SIMD array processing system with routing networks having plurality of switching stages to transfer messages among processors

ABSTRACT

An array processing system including a plurality of processing elements each including a processor and an associated memory module, the system further including a router network over which each processing element can transfer messages to other random processing elements, a mechanism by which a processor can transmit data to one of four nearest-neighbor processors. In addition, the processing elements are divided into groups each with four processing elements, in which one of the processing elements can access data in the other processing elements&#39; memory modules. The routing network switches messages in a plurality of switching stages, with each stage connecting to the next stage through communications paths that are divided into groups, each group, in turn being associated with selected address signals. A communications path continuity test circuit associated with each path detects any discontinuity in the communications path and disables the path. Thus, the stage may attempt to transfer a message over another path associated with the same address.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates generally to the field of digital data processingsystems, and more particularly to array processing systems whichincorporate a large number of processors that are interconnected in aregular connection structure and in which all of the processors receivethe same instruction from a common control structure.

2. Description of the Prior Art

A digital data processing system comprises three basic elements, namely,a memory element, an input/output element and a processor element. Thememory element stores information in addressable storage locations. Thisinformation includes data and instructions for processing the data. Theprocessor element fetches information from the memory element,interprets the information as either an instruction or data, processesthe data in accordance with the instructions, and returns the processeddata to the memory element. The input/output element, under control ofthe processor element, also communicates with the memory element totransfer information, including instructions and data to be processed,to the memory, and to obtain processed data from the memory.

Most modern data processing systems are considered "von Neumann"machines, since they are constructed according to a paradigm attributedto John von Neumann. Von Neumann machines are characterized by having aprocessing element, a global memory Which stores all information in thesystem, and a program counter which identifies the location in theglobal memory of the instruction being executed. The processing elementis executing one instruction at a time, that is, the instruction that isidentified by the program counter. When that instruction has beenexecuted, the program counter is advanced to identify the location ofthe next instruction to be executed. (In most modern systems, theprogram counter is actually advanced before the processor has finishedprocessing the current instruction).

Von Neumann systems are conceptually uncomplicated to design andprogram, since they do only one operation at a time, but they are alsorelatively slow. A number of advancements have been made to the originalvon Neumann paradigm to permit the various parts of the system, mostparticularly the various components of the processor, to operaterelatively independently and achieve a significant increase inprocessing speed. The first such advancement was pipelining of thevarious steps in executing an instruction, including instruction fetch,operation code decode (a typical instruction includes an operation codewhich identifies the operation to be performed and in most cases one ormore operand specifiers which identify the operands, or data, to be usedin executing the instruction), operand fetch, execution (that is,performing the operation set forth in the operation code on the fetchedoperands), and storing of processed data, which are performed relativelyindependently by separate hardware in the processor. In a pipelinedprocessor, the processor's instruction fetch hardware may be fetchingone instruction while other hardware is decoding the operation code ofanother, fetching the operands of another, executing yet anotherinstruction and storing the processed data of a fifth instruction.Pipelining does not speed up processing of an individual instruction,but since the processor begins processing a second instruction before ithas finished processing the first, it does speed up processing a seriesof instructions.

Pipelining has also been used within several of the circuits comprisingthe processor, most notably the circuits which perform certainarithmetic operations, to speed processing of a series of calculations.Like pipelining of instruction processing, pipelining arithmeticoperations does not speed up an individual calculation, but it doesspeed up processing of a series of calculations.

A pipelined processor is obviously much more complicated than a simpleprocessor in a von Neumann system, as it requires not only the variouscircuits to perform each of the operations (in a simple von Neumannprocessor, many circuits could be used to perform several operations),but also control circuits to coordinate the activities of the variouscircuits. However, the speed-up of the system can be dramatic.

More recently, some processors have been provided with executionhardware which include multiple functional units each being designed toperform a certain type of mathematical operation. For example, someprocessors have separate functional units for performing integerarithmetic and floating point arithmetic, since floating pointarithmetic requires handling two parts of a floating point number,namely the fraction and the exponent, while numbers in integerarithmetic have only one part. Some processors, for example the CDC 6600manufactured by Control Data Corporation, included a number of separatehardware functional units each of which performs one or only severaltypes of mathematical operations, including addition, multiplication,division, branch, and logical operations, all of which may be executingat once. This can be helpful in speeding up certain calculations, mostparticularly those in which several functional units may be used at onetime for performing part of the calculation.

In a processor which incorporates pipelining or multiple functionalunits (or both, since both may be incorporated into a processor), asingle instruction stream operates on a single data stream. That is,each instruction operates on data to produce one calculation at a time.Such processors have been termed "SISD", for "single instruction-singledata". However, if a program requires a segment of a program to be usedto operate on a number of diverse elements of data to produce a numberof calculations, the program causes the processor to loop through thatsegment for each calculation. In some cases, in which the programsegment is short or there are only a few data elements, the timerequired to perform the calculations on the data is not unduly long.

However, for many types of such programs, SISD processors would requirea very long time to perform all of the calculations that are required.Accordingly, processors have been developed which incorporate a largenumber of processing elements, all operating concurrently on the sameinstruction, with each processing element processing a separate datastream. These processors have been termed "SIMD" processors, for "singleinstruction-multiple data".

SIMD processors are useful in a number of applications, including imageprocessing, signal processing, artificial intelligence, databaseoperations and computer simulation of a number of things such aselectronic circuits and fluid dynamics. In image processing, eachprocessor performs processing on a pixel ("picture element") to enhancethe overall image. In signal processing, the processors concurrentlyperform a number of the calculations required to produce the FastFourier Transform of the signal. In artificial intelligence, theprocessors perform searches on extensive databases representing thestored knowledge of the application. In database operations, theprocessors perform searches, as in the artificial intelligenceapplications, and they also perform sorting operations. In computersimulation of, for example, electronic circuits, each processorrepresents one part of the circuit, and the processor's calculationsindicates the response of the part to signals from other parts of thecircuit. Similarly, in simulating fluid dynamics, which can be useful ina number of applications such as weather prediction and the design ofairplanes, each processor is associated with one point in space, and thecalculations performed provide information about various factors such asfluid flow, temperature, pressure, and so forth.

Typical SIMD processors include two primary components, namely an arrayof processor elements and a routing network over which the processorelements may communicate the results of a calculation to other processorelements for use in future calculations. In addition, SIMD processorsinclude a control processor for controlling the operations of theprocessor elements and routing network in response to instructions anddata from a host computer system.

Several routing networks have been used in SIMD processors and a numberof others have been proposed. In one routing network, which has beenused in the Massively Parallel Processor, manufactured by GoodyearArrowspace Corporation ("Goodyear MPP"), the processor elements areinterconnected in a matrix, or mesh, arrangement. In such anarrangement, the processor elements are connected in rows and columnsand directly communicate only with their four nearest neighbors. Thisarrangement can be somewhat slow if communications may be to randomprocessor elements, but the number of wires which are required to makethe interconnections is lower than in most other arrangements, on theorder of 4n, where "n" is the number of processor elements, assumingonly unidirectional transfer of messages over each wire. If each wirecan transfer bidirectionally, the number of wires is reduced by half,with a possible reduction in the message transfer rate.

The matrix network is also used on the "Connection Machine",manufactured by Thinking Machines Corporation, but that machine alsoincludes a hypercube network allowing communications between randomprocessor elements (that is, processor elements which are not nearestneighbors). In a hypercube network, each processor chip connectsdirectly to twelve other processor chip. Each processor chip includesseveral processor elements and circuits which form part of the routingnetwork. The routing circuits on each chip receive messages from theprocessor elements on the chip for transmission to processor elements onother processor chips. In addition, the routing circuits receivemessages from other processor chips. If a message from another processorchip is to be received by a processor element on the chip, it forwardsit to that element; however, if the message is to be received by aprocessor element on another chip, it transmits the message over a wireto another processor another chip. The procedure is repeated until themessage reaches the intended recipient. Thus, the routing circuits oneach chip must be able to handle not only messages from the processorelements on the chip, but also from messages from other chips which mayor may not be addressed to processor elements on the chip.

A hypercube network handles communications fairly rapidly, but itrequires a large number of wires, on the order of 12n, if messages aretransferred unidirectionally over each wire. For example, if "n" were4096 (4K, K=1024), the hypercube would require on the order of 48Kwires. If the wires transfer messages bidirectionally, only 24K wireswould be required, but the volume of message traffic that could becarried would also be reduced. Typically, the larger the number of wiresin a routing network, the more expensive is the network, and the greateris the likelihood of failure resulting from broken wires or failedswitching elements which interconnect the wires.

Another routing arrangement which has been proposed is a crossbarswitch, through which each processor element can communicate with any ofthe other processor elements directly. The crossbar switch provides themost efficient communications of any of the routing networks proposed.However, a crossbar switch also has the most wires and switchingelements, both on the order of n², and thus is most expensive and mostsusceptible to failure due to broken wires and switching elements. Usingthe above example, in which "n" is 4K, the number of wires and switchingelements required for the crossbar switch is 16M (M-1,048,576).

Yet another routing arrangement is an omega network, in which switchingis performed through a number of serially-connected stages. Each stagehas two inputs, each connecting to the outputs of two prior stages orprocessor chips and has two outputs. The "Butterfly" manufactured byBolt Beranek and Newman, use an omega network.

The cost of a routing network is directly related to the number ofwires, as is the likelihood of failure due to discontinuity in acommunications path. To reduce the number of wires and achieve asignificant fraction of the efficiency of the crossbar switch, a routingnetwork has been proposed in which a multiple-stage omega networkperforms some portion of the switching. The output from the omeganetwork is connected to a crossbar switch, which would require manyfewer switching connections than would be necessary in the absence ofthe omega network. Depending on the number of stages in the omeganetwork, the number of wires may be less than in a hypercube, while thetransfer efficiency would be greater than a hypercube. For example, if"n" is 4K, and a seven-stage omega network is provided to a crossbarswitch, 36K wires (again assuming unidirectional communications overeach wire) would be required to form the routing network.

Using a routing network to transfer data does have a number oflimitations. The mesh network is useful generally when transferring dataonly to the adjacent processors, as each transfer requires commands fromthe controlling program. A hYpercube, crossbar switch, omega, or likenetwork is most useful if message transfers are expected to be to randomprocessors. Some array processors, the Thinking Machine, for example,have two mechanisms for transferring data, one for random transfers andthe other for matrix transfers. Under some circumstances, however, itmay be faster to provide a processor with direct access to memoriesassociated with other processing elements. This may be useful, forexample, when, after performing operations in parallel, a serialoperation is to be performed on the just processed data. If oneprocessing element has access to the data in at least some otherprocessing elements' memories, the processor may perform serialoperations using that data. Also, the processing element may use thosememories to if a problem requires more storage capacity than a singleprocessing element would have.

Typically, array processors are used in performing arithmetic operationson numerical values, which are expressed in "floating point" form. Inthat form, a floating point number has a fraction portion and anexponent portion, with the value of the number being the value containedin the fraction portion multiplied by the value two raised to the valuecontained in the exponent portion. When performing arithmetic operationssuch as addition and subtraction on such numbers, the numbers must be"aligned", that is, they must have the same value of the exponent. Toachieve this, the value of the fraction portion of the floating pointnumber must be reduced, which raises the effective value of the number'sexponent portion, until the exponent values are equal. After thearithmetic operation, the fraction of the result must be normalized,that is, leading zeroes must be removed by decreasing the value of thefraction portion, while at the same time increasing the value of theresult,s exponent. In both the alignment and normalization operations,the fractions are reduced and or increased by shifting their values inthe locations in which they are stored.

However, in the alignment and normalization operations, since the valuesof the numbers processed by the various processing elements are alldifferent, the number of shifts required to effect the alignment ornormalization will also be different.

SUMMARY OF THE INVENTION

The invention provides a new and improved array processing system havinga plurality of processing elements each with a processor and anassociated memory.

In one aspect, the invention provides an array processing systemincluding a plurality of processing elements each including a processorand an associated memory, in which the processing elements are dividedinto groups each having a selected number of processors and associatedmemories. In response to control signals, one processor in each group isenabled to obtain data from any of the memories associated with any ofthe processors in the group. While operating in this mode, the otherprocessors are effectively disabled. This aspect allows computations tobe performed in parallel on data in the various memories by all of theprocessors in the group, and then facilitates the serial processing ofthat data by one of the processors in the group, without requiring theother processors to transfer the data to the one processor over theinterconnection mechanism. In addition, it permits the array to providelarger amounts of memory, using smaller numbers of processors, ifnecessary to perform a computation.

In another aspect, the invention provides an array processor whoseprocessing elements can process data a bit at a time, in bit mode, or anibble (four bits) at a time, in nibble mode.

In another aspect, the invention provides a processor having a shiftregister that can shift both toward a shift out terminal and toward ashift in terminal, and that can shift toward the shift out terminal abit at a time or a nibble (four bits) at a time. The ability to shift anibble at a time facilitates the operation in nibble mode. The abilityto shift toward the shift in terminal is useful in floating pointarithmetic, particularly in equalizing the exponent portion of a numberprior to addition or subtraction, and in normalizing the fractionportion of a number after an arithmetic operation.

In yet another aspect, the invention provides a mechanism for quicklyand easily generating a status signal for indicating when all of theprocessors in the array have completed an operation such asnormalization following arithmetic operations or transmission ofmessages.

In still another aspect, the invention provides an array processingsystem in which the processing elements can receive data from, andtransmit data to, other processing elements in a matrix arrangement, andin which they can provide that the data transmitted is the logical OR ofthe received data and the processing element's own data, to effect alogical OR of the data being transmitted by all of the processingelements along an entire row or column.

In a further aspect, the invention provides an array procssing systemhaving a message transfer mechanism that permits one processing element,which might otherwise have to serially transmit messages to a largenumber of other processing elements, to use other processing elements asassistants in transmitting messages to the other processing elements,thereby reducing the amount of time required to transmit all of thenecessary messages.

In a further aspect, the invention provides an array processing systemincluding a routing network for transferring messages, with each messagecontaining an address identifying the intended recipient. The routingnetwork includes a plurality of switching stages each of which uses aselected number of bits of the address to identify a switching paththrough each stage. The lines connecting between stages are divided intogroups, with each group including a plurality of lines each associatedwith one encoding of the address bits used by the stage in selecting theswitching path. Each switching stage can couple a message having therequired address encoding onto any one of the lines in the appropriategroup.

In another aspect, the invention provides an array processing systemincluding a routing network in which messages are transferred overcommunications path, with the routing network including a system fordetecting when a communications path has become discontinuous, and forthereafter not using that path.

In yet a further aspect, the invention provides an array processingsystem including routing network for transferring messages over acommunications path, with the routing network including a system fordetecting when a communications path has become discontinuous, and forthereafter not using that path. Each message contains an addressidentifying the intended recipient. The routing network includes aplurality of switching stages each of which uses a selected number ofbits of the address to identify a communications path through eachstage. The communications paths connecting between stages are dividedinto groups, with each group including a plurality of paths eachassociated with one encoding of the address bits used by the stage inselecting the switching path. Each switching stage can couple a messagehaving the required address encoding onto any one of the paths in theappropriate group for transmission to the next stage. If onecommunications path in a group is not used because it is discontinuous,the stage may transmit a message over any of the other communicationspaths in the group to the next stage.

In yet another aspect, the invention provides an array processing systemhaving a multiple-stage routing network, in which the first stage isenabled to begin transferring a message, and in which each stage enablesthe next stage when it is time for that stage to begin transferring themessage.

BRIEF DESCRIPTION OF THE DRAWINGS

This invention is pointed out with particularity in the appended claims.The above and further advantages of this invention may be betterunderstood by referring to the following description taken inconjunction with the accompanying drawings, in which:

FIG. 1 is a general organizational diagram depicting the generalcomponents of an array processing system, including a processor arrayconstructed in accordance with the invention;

FIG. 2 is a more detailed block diagram of the processor array depictedin FIG. 1, depicting one mechanism for transferring information throughthe array;

FIG. 3 is another more detailed block diagram of the processor arraydepicted in FIG. 1, depicting another mechanism for transferringinformation through the array;

FIG. 4 is a detailed block diagram of a processor element forming partof the processor array depicted in FIG. 1;

FIG. 5A is a block diagram depicting the interconnections betweenseveral processor elements and memory elements in the processor arraydepicted in FIG. 1,

FIG. 5B is a schematic circuit diagram depicting a circuit of a portionof the memory control circuit depicted in FIG. 5A;

FIG. 5C is a schematic circuit diagram of a circuit of the memorycontrol circuit associated with 1 PE depicted in FIG. 5A;

FIG. 6 is a block diagram depicting generation of various status signalson each processor chip used by the array control unit depicted in FIG.1.;

FIG. 7 is a detailed circuit diagram of a shift register which formspart of the processor element depicted in FIG. 4;

FIG. 8A is a detailed block diagram depicting the routing network usedin the information transfer mechanism depicted in FIG. 2 and FIG. 8Bdepicts the detailed structure of a message transferred through therouting network depicted in FIG. 8A;

FIGS. 9-1 and 9-2 together are a detailed block diagram depicting theorganization of a switching chip used in the routing network depicted inFIG. 8A; and

FIG. 10A is a detailed circuit diagram of an input circuit of FIG. 9-1and 9-2;

FIG. 10B-1 and 10B-2 are detailed circuit diagrams which together depicta circuit of an address control circuit of FIG. 9-1 and 9-2.

DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

I. General Description Of Array Processor

With reference to FIG. 1, an array processor constructed in accordancewith the invention includes three major sections, including an arraycontrol unit 10, a processor array -1 and a system interface 12. Thecontrol unit 10 receives data and instructions from a host dataprocessing system (not shown) through the interface 12, transmits thedata to the array 11 over a bus 13 and issues to the array 11microinstructions over a bus 14. The microinstructions enable the array11 to process the data in accordance with the host's instructions.

During execution of the microinstructions, the processor array 11 maytransmit various status signals back to the array control unit over astatus bus 15. These status signals are used by the array control unit10, as described below, in the selection of the microinstructions to beissued to the processor array. After the processor array performs all ofthe operations responsive to the microinstructions from the arraycontrol unit in processing the instruction from the host data processingsystem, the control unit 10 then enables the processing array 11 totransmit the processed data to the array control unit 10 over bus 13,and the array control unit may then transmit it to the host throughsystem interface 12.

II. General Organization of Processor Array 11

Processor array 11 includes two general portions, namely a large numberof processor elements (PE's) on a plurality of processor element (PE)chips, and a mechanism for transferring information to and from thearray control unit 10 and among the processor elements. In one specificembodiment, each PE chip includes thirty-two processing elements. Asdescribed in more detail below in connection with FIG. 4, each processorelement includes a processor, including circuitry for performingselected arithmetic and logical operations, and an associated memory. Inthe aforementioned specific embodiment, each memory has a capacity of1024 bits (1 Kb).

The PE's have two mechanisms for transferring information to otherprocessors. In one mechanism, a "random transfer" mechanism, which isgenerally depicted in FIG. 2, each PE can send a message to any otherPE. In the second mechanism, a "nearest neighbor" mechanism, which isillustrated in FIG. 3, the PE's are interconnected in a "nearestneighbor" mesh manner with four other PE's, and each PE may sendinformation only to one of those four PE's to which it is directlyconnected. In the random transfer mechanism, communications aremaintained over a routing network 30 (see FIG. 2), which is describedbelow connection with FIG. 2 and in more detail in connection with FIGS.8A through 10C. Circuits for implementing the second mechanism aredepicted in FIG. 4.

With reference to FIG. 2, a plurality of PE chips 20A through 20N(generally identified by reference numeral 20) are depicted. Each PEchip 20 has a plurality of PE's. Since the PE chips are similar, onlychip 20A is shown in detail. PE chip 20A includes a plurality of PE's21A through 21M in an ordered heirarchy. Each PE includes a processor22A through 22M and a memory 23A through 23M. When a PE is enabled totransmit a message, it's TR transmit request flag 24A through 24M isset. When a PE's RTR OUT EN IN router out enable in signal is asserted,the PE is enabled to transmit its message through a router outputcontrol circuit.

The RTR OUT EN IN router out enable in signal is a daisy-chained enablesignal. When it becomes the PE's turn to send a message, which occurswhen its RTR OUT EN IN router out enable in signal is asserted, ittransmits its message signals through a router output control circuit25, which transmits the message from the PE chip 20A as a CHP 0 MSG OUTchip (0) message out signal. An ACK acknowledge flag 26A through 26M,which initially is cleared, is set when a message has been sent andacknowledged. A benefit of providing separate TR transmit request andACK acknowledge flags, rather than merely having the TR transmit requestflag reset in response to the acknowledgement, will be made clear below.

After a PE's message has been transmitted and acknowledged, the PE 22asserts the RTR OUT EN IN router out enable in signal for the next PE.If that PE's TR transmit request flag 24 is set, and the ACK acknowledgeflag 26 is reset, that PE is enabled to transmit a message. Either ifthe TR transmit request flag 24 is cleared, or if the TR transmitrequest flag 24 is set but the ACK acknowledge flag 26 is also set, thePE does not transmit a message, but instead asserts the RTR OUT EN INrouter out enable in signal for the next PE 22. After messages have beensent and acknowledged for all of the PE's whose TR transmit requestflags 24A through 24M were initially set, the control circuit 25generates a LOCAL DONE signal, which is ORed with LOCAL DONE signalsfrom other PE chips to form a DONE signal. The DONE signal istransmitted over status bus 15 to the array control unit 10, to indicatethat all PE's have transmitted messages.

In one specific embodiment, to reduce the number of input terminals tothe routing network 30, the PE chips 20A through 20N are paired in theirtransmission of messages through the routing network. With reference toFIG. 2, the last PE 22(m) on PE chip 20A provides the RTR OUT EN INrouter out enable in signal to the first PE 22 on PE chip 20B. Thus,after all message have been transmitted from PE chip 20A andacknowledged, PE chip 20B begins transmitting messages.

The message outputs of selected ones of the PE chips 20A through 20N areconnected to multiplexers 32 which selectively couple the outputs of thePE chips or the data signals from the data bus 13 to the routingnetwork. The routing network 30 thus facilitates the transfer of datafrom the array control unit 10 into the processor array 11. Morespecifically, with reference to PE chip 20A, the output of circuit 25 ofPE chip 20A is connected to one input terminal of a multiplexer 32. Theother input terminal of multiplexer 32 is connected to one line of databus 13. The multiplexer is controlled by a SEL DB select data bus Signalwhich, when asserted, enables the multiplexer 32 to couple the signalson the line of data bus 13 to the routing network 30. When the SEL DBselect data bus signal is not asserted, the multiplexer 32 couples thesignals from the PE chips to the routing network 30.

The output signals from the routing network 30 (the routing network willbe described below in connection with FIGS. 8A through 10C) are coupledto message input terminals on the PE chips 20A through 20N, andspecifically to a router input control circuit 33 shown in PE chip 20A.The router input control circuit first determines whether the or not themessage is for a PE on the PE chip on which it resides. Thisdetermination is made in response to the first signal received by therouter input control circuit and by the condition of a HI 32 signal. Asshown in FIG. 2, the HI 32 signal is transmitted in uncomplemented formto one of each pair of PE chips, such as PE chips 20A and 20(N-1), andin complemented form to the other PE chips, such as PE chips 20B and20N. If the router input control circuit 33 receives the uncomplementedHI 32 signal it accepts the messages for which the first signal isasserted, but if the router input control circuit 33 receives thecomplemented HI 32 signal, it accepts the messages for which the firstsignal is negated. The next few signals, as explained below inconnection with FIG. 8B, identify the PE 22 which is the intendedrecipient of the message, and the router input control circuit directsthe message to the PE 22 so identified.

The output signals from selected ones of the lines from routing network30 are also coupled to data bus 13 to carry data output signals to thearray control unit 10.

It will be appreciated by the foregoing description that, while a numberof messages may be transferred through routing network 30 at one time,which will be termed herein a "minor message cycle", a number of minormessage cycles may be required to transfer all messages that arerequired to be transmitted at one time in response to a programrequirement. In particular, at most one PE 22 may transmit a message ina PE chip pair at one time, so if more than one PE 22 on a PE chip pairmust transmit a message, at least that number of minor message cyclesare required. In addition some messages may be blocked because ofcontentions in routing network 30 or because multiple messages are beingtransmitted to the same PE chip pair during a minor message cycle.

As has been noted, providing a separate TR transmit request flag 24Athrough 24M and ACK acknowledge flag 26A through 26M to control messagetransfer has a number of benefits. In particular, as described below, inone specific embodiment each message includes a header containing anaddress which identifies the PE to receive the message and anacknowledgement timeslot for the receiving PE, or more specifically thePE chip 20 containing the receiving PE, to transmit a signal to thetransmitting PE indicating receipt of the message. Following the messageheader, the message data is sent. Placing the acknowledgement timeslotin the header rather than at the end of the message simplifies timing ofthe acknowledgement time slot, since the header is of fixed lengthwhereas the length of the message data may vary as between messages.

In some applications of the array processor, it is necessary for one PEto send the same message to a number of different PE's. In sending suchmessages, the transmitting PE iteratively sends the message to eachintended recipient. If a large number of PE's are to receive themessage, this procedure can take a long time. This time may beconsiderably reduced if the originating PE first transmits the messageto a selected number of other "assistant" PE's, which then, along withthe originating PE, send the messages to other PE's. This processcontinues until all of the intended recipient PE's receive the messages.

At some point during or immediately after the transfer of a message fromthe originating PE to an assistant PE, it will be necessary for thearray control unit 10 (FIG. 1) to transmit control signals to array 11to enable the setting of the TR transmit request flags so as to enablethe assistant PE's to transmit the messages and, if necessary, to enablethe originating PE to send messages to other PE's, which may beassistant PE's or final destination PE's. If, during the originalmessage, the acknowledgement of the receipt of the original messageenabled the TR transmit request flag 24A through 24M to be reset, the PEwould be unable to determine whether the original message had been sentto the first assistant PE, and it would retransmit the message to thatPE. Thus, providing separate TR transmit request and ACK acknowledgeflags 24A through 24M and 26A through 26M, respectively, facilitates theuse of assistant PE's to transmit messages in a "spreading activation"fashion.

The "nearest neighbor" message transfer mechanism is depicted in FIG. 3.With reference to FIG. 3, a plurality of PE's 20A through 20N aredepicted organized in an array pattern of a plurality of "k" rows and"1" columns, where N-k1. Each PE 20A through 20N can transmitinformation, in the form of signals, to its four nearest neighbors inthe array. Specifically, for example, the PE 20(k+2) can transmitsignals "westwardly" to PE 20B and "northwardly" to PE 20k+1 (shown onFIG. 3), and "southwardly" to PE 20(k+3) and "eastwardly" to 20(2k+2) ina row and column not shown in FIG. 3. The PE's in the leftmost column,as shown in FIG. 3, that is PE's 20A through 20K, can transmit signalswestwardly to the rightmost column as shown in FIG. 3, that is, to PE's20(1k+1) through 20N, and the PE's in the rightmost column can transmitsignals eastwardly to the PE's in the leftmost column. Similarly, thePE's in the top row, that is, PE's 20A, 20(k+1) . . . 20(1k+1), cantransmit signals northwardly to the PE's in the bottom row, that is PE's20K, 20(2k) . . . 20N, and the PE's in the bottom row can transmitsignals southwardly to the PE's in the top row. The mechanism used bythe PE's to transmit in the "nearest neighbor" mode will be describedbelow in connection with FIG. 4.

The nearest neighbor message transfer mechanism may also be used forinput of signals, representing data, to the array 11 or output of datafrom array 11. In particular, with reference to FIG. 4, a multiplexernetwork (not shown) may be connected at the left side of the array, forexample to permit data to be loaded into the leftmost column of PE'sdata from either the rightmost column or from an external data source.The data is initially latched by the leftmost column and is transmittedto the next column, that is PE's 22(K+1) through 22(2K) as more data islatched by the leftmost column. That process continues until data hasbeen passed throughout the entire array.

The output of data is effectuated by means of the nearest neighbortransfer mechanism by providing a set of output drivers (not shown)which are connected to a set of appropriated receiving and recordingcircuits. The output drivers may be connected to, for example, theeastward transmission terminals of the rightmost column of the array toreceive data signals from the lines which also connect the PE's alongthat column to the PE's on the leftmost column. To allow data to beretrieved from the array 11, the PE's are enabled to transmit the datain an easterly direction while recording the outputs of the outputdrivers. After the data from the leftmost column has returned to it,data from all of the PE's in the array will have passed through therightmost column of PE's and have been transmitted by the drivers.

III. Processing Element

A. General Description (Discussion of FIG. 4)

With reference to FIG. 4, a PE constructed in accordance with theinvention includes a full adder 51 which receives input signals from aone of a pair of shift registers, namely A shift register 52 or B shiftregister 53, from a Boolean logic circuit 54, and from a carry latch 55.A multiplexer 56 selects the output signals from the S/O shift outoutput terminals of one of the A or B shift registers 52 or 53, inresponse to an A/B select signal representing a bit in the controlmicroword received from the array control unit 10 over bus 14 (FIG. 1).As is typical in an array processing system, during any given operatingcycle, as governed by one or more clock signals (not shown), a singlecontrol microword is transmitted by the array control unit 10 to all ofthe PE's in the array 11 to control the operations of the PE's in thearray 11 during that cycle. Depending on the condition of one or moreflags, as described below, a PE may or may not perform the operationsenabled by the control microword.

If a CLO clear shift register output signal from the control microwordis negated, the multiplexer is enabled to couple the output signal fromthe selected shift register to the connected input of the adder 51. Byasserting the CLO signal, the array control unit enables the multiplexerto couple an output signal having an effective value of zero to theinput of adder 51. This provides a mechanism for loading a value into ashift register prior to performing an arithmetic operation.

The A and B shift registers 52 and 53 operate in response to a number ofcontrol signals from the control microword. Both the A and B shiftregisters are of variable effective length, that is, the output bitposition from which output signals are transmitted to multiplexer 56 canbe varied in response to A SHFT SZ (A shift register size) and B SHFT SZ(B shift register size) signals from the control microword. In oneembodiment, the A shift register 52 has effective lengths of four,eight, twelve, sixteen, twenty, twenty-four, twenty-eight and thirty-twobits, and may be used for performing integer arithmetic and arithmeticon the fraction portion of floating point numbers. In the sameembodiment, the B shift register 53 has effective lengths of four, eightand twelve bits, and may be used for performing arithmetic on theexponent portion of floating point numbers.

The contents of the A and B shift registers 52 and 53 are shifted inresponse to a SHFT EN shift enable signal generated by an OR gate 60.The OR gate 60 is energized by an SH shift signal from the controlmicroword, and by an asserted signal from an AND gate 61. The AND gateis enabled by an M signal from an M flag 62 and is energized by an MSH(M shift) signal from the control microword. Thus, the array controlunit may enable an unconditional shift by asserting the SH shift signal,or it may alternatively enable a shift conditioned on the state of the Mflag 62 by asserting the MSH signal in the control microword

The PE's operate in either a bit serial mode or a nibble mode, in whicha nibble (four bits) is processed during a control cycle. The PEoperates in nibble mode when an NM nibble mode signal from the controlmicroword is asserted. When the NM nibble mode signal is asserted, the Aand B shift registers 52 and 53 shift four bits in a cycle, otherwisethey shift one bit in a cycle. Also, when in nibble mode, the A and Bshift registers receive and store four bits in a cycle from their shiftin input terminals (designated S/I on the Figure).

Finally, as described in detail below in connection with FIG. 7, the Ashift register 52 also can shift in both directions, that is, toward theS/O output terminal or toward the S/I shift in input terminal. The shiftdirection is selected in response to the SHFT DIR shift direction signalfrom the control microword. The ability to shift the contents of theshift register toward the S/I input terminal is useful in floating pointarithmetic, for example, particularly in normalization, that is, inremoving leading zeroes, which are stored in the end of the shiftregister toward the S/I shift in input terminal.

The Boolean logic circuit 54 couples data signals to the full adder 51from a four-line data bus 58 or from one of the nearest neighbor PE's.Depending on the state of LGC FTN logic function signals from thecontrol microword, the Boolean logic circuit 54 may perform one ofsixteen logical operations between an X flag 63 and signals from thedata bus 58 or from the nearest neighbor PE's. If a logical operation isperformed between the X flag 63 and the signals on the data bus 58, thelogical operation is performed with respect to the signal on each lineof the data bus 58 individually. For example, if an AND operation is tobe performed, four AND operations are actually performed, one betweenthe X flag 63 and the signal on each of the lines of the data bus 58.

In either bit-serial or nibble mode, the carry latch 55 supplies a onebit CARRY signal to the full adder 51. The carry latch 55 is latches theCARRY signal whenever the SHFT EN shift enable signal from OR gate 60asserted. A CL C clear carry signal from the control microword, whenasserted, forces the output of the carry latch 55 to zero. The CL Cclear caryy signal is useful for the same purpose as the CLO clear shiftregister output signal, namely, initial loading of the carry latch 55prior to performing an addition operation.

Furthermore, in either bit serial or nibble mode, the full adder 51generates four ADD OUT (3:0) adder output signals and a C OUT carry outsignal. The ADD OUT (3:0) adder output signals are coupled to the S/Ishift in terminals of the A and B shift registers 52 and 53 and to aninput terminal of an adder latch 57. The C OUT carry out signalrepresents the carry signal from the high-order adder stage of the fulladder 51, that is, the stage that generates the high-order ADD OUT (3)adder output signal.

The C OUT signal and the C OUT (0) carry signal from the low order stageof the adder, which represents the carry from that stage, are coupled toa multiplexer 64 which, in response to the NM nibble mode signal fromthe control microword, couples one of the signals to an input of asecond multiplexer 68. Multiplexer 68, in turn, in response to a LD Cload carry signal from the control microword, selects either the CARRYOUT signal from multiplexer 64 or the LOG OUT (0) logic output (0)signal from Boolean logic circuit 54 for storage in the carry latch 55.If the LD C load carry signal is asserted, the multiplexer 68 isconditioned to couple the LOG OUT (0) logic output (0) signal to thecarry latch 55 and the carry latch is enabled to latch the signal. Thisfacilitates initializing the carry latch 55 to a known state fromthrough Boolean logic 54. If, on the other hand, the LD C load carrysignal is negated, the multiplexer 68 is conditioned to couple the CARRYOUT signal from multiplexer 64 to the input of the carry latch. TheCARRY OUT signal is latched if the SHFT EN shift enable signal from thecontrol microword is asserted.

Thus, in nibble mode, the carry latch 55 receives the carry from thehigh-order stage of the full adder 51, representing the carry from thehigh order stage of the nibble. In bit-serial mode, the carry latch 55receives the carry from the low-order stage of the adder. In bit-serialmode, the arithmetic operations in full adder 51 are essentially carriedout in the low-order stage, as the signals from the higher-order stagesare effectively ignored.

The M flag 62 is also conditioned by signals from the full adder 51through a multiplexer 65. The multiplexer couples either the ADD OUT (0)signal from the full adder 51, or, if the NM nibble mode signal isasserted, the ADD OUT (3) signal. The signal from the multiplexer isinverted by an inverter 66 and the result coupled to an input terminalof a second multiplexer 67. The multiplexer 67 also includes a secondintput terminal which receives the low-order signal from the Booleanlogic circuit 54. If an NSM "not sum to M" signal from the controlmicroword is asserted, the multiplexer 67 couples the complement of thehigh-order signal from the full adder 51 to the M flag 62 for storage inresponse to an LD M load M signal from the control microword. Otherwise,the M flag 62 is loaded from the signals from the Boolean logic circuit54. An AND gate 70 couples the condition of the M flag 62 to data bus58, particularly the low-order DATA (0) line thereof, in response to aRD M read M signal from the control microword.

If the M flag 62 is loaded from the full adder 51, it may be used, forexample, in normalization during floating point arithmetic. In thisoperation, the fraction portion of the floating point number, which isstored in the A shift register 52, is shifted toward the S/O shift outterminal (that is, the high-order end) until the high-order stagecontains the value "one". In response to each shift of the contents ofthe A shift register 52, the exponent, which is in the B shift register53, is incremented. Since different PE's in array 11 may requiredifferent numbers of shifts to normalize the contents of theirrespective A shift registers 52, the shift operations then depend on thestate of the M SH (M conditioned shift) signal from the controlmicroword. Until a "one" reaches the high-order stage of the A shiftregister 52, the inverter 66 conditions the M flag 62 to be set (thatis, store the complement of the zero value from the shift register 52),and thus the M signal is asserted. Thus, the M SH signal enables the Aand B shift registers 52 and 53 to operate. However, when a "one"reaches the high-order stage of the A shift register 52, the inverter 66conditions the M flag to be clear, negating the output M signal. Whenthe M SH (M conditioned shift) signal from the control microword isasserted, the negated M signal disables the shift registers 52 and 53.The M flag 62 can also be used for initial alignment of floating pointnumbers prior to an addition or subtraction in a similar manner.

Two additional flags are loaded from the low-order output signal fromBoolean logic circuit 54, namely a WRT memory write enable flag 71 andthe TR transmit request flag 24. The WRT memory write enable flag 71 isloaded in response to an LD WRT load memory write enable signal from thecontrol microword and the TR transmit request flag 24 is loaded inresponse to an LD TR load transmit request signal from the controlmicroword. The WRT memory write enable flag 71 generates a WRT memorywrite signal which is coupled to the PE chip's router control circuit 33which, in turn, generates a WRT EN write enable signal associated witheach of the memories 23 on the chip to control writing of data thereto.

The router control circuit 33 enables writing data to the memory underthree circumstances. Under normal operation, that is, when the routingnetwork 30 (FIG. 2) is not in use, the router control circuit 33generates the WRT EN write enable signal for the memory when the WRwrite signal from the control microword is asserted if the associatedPE's WRT flag is set. This enables the associated memory 23 to store thedata bits from the data bus 58 through transceiver 72 in the locationspecified by the address signals in the control microword.

Alternatively, when the routing network is in use and actuallytransferring data, the router control circuit 33 receives the data as aMSG IN message in signal. Contemporaneously, the ENR enable routersignal is asserted. As will be explained below in connection with FIG.10A, the first six bits received by the PE chip 20 identify the chip ofthe chip pair, and PE on the chip, to receive the message data. Therouter control circuit 33 receives the PE identification portion of theMSG IN message in signals, transmits the data portion of the MSG INmessage in signals as a DATA (0) signal onto the data bus 58 of theintended recipient PE and asserts the WRT EN write enable signal of theassociated recipient.

Finally, the router control circuit 33 also enables the memory 23 tostore data when the routing network is in use but between messages. Thismay occur when, for example, several messages are to be directed to asingle PE by several PE's during one major message transfer cycle, andbetween messages the recipient PE is to perform a calculation on thejust received data. As explained above, a major message transfer cyclecomprises a plurality of minor message transfer cycles each initiated bythe ENR enable router signal and terminated by a BND branch if not donesignal. If the DONE signal is not asserted at the end of a minor messagetransfer cycle, some PE's have messages yet to transfer, and the BNDsignal from the control microword indicates that the array control unitwill return to the beginning of the message transfer sequence. If acalculation is to be performed before the beginning of the next minormessage transfer cycle, the ENR enable router signal is negated, but theBND branch if not done signal is delayed until the calculation isperformed and the results written into memory 23. In this case, thememory 23 is enabled by the router control circuit 33 if the WRT flag isset and the WR write signal from the control microword is asserted.However, the WRT EN write enable signal will be asserted only for thosePE's which just received a message during the current minor messagetransfer cycle.

The data output from memory 23 is coupled to a memory data outputcontrol circuit 73 described below in connection with FIGS. 5A and 5B.In brief, the memory data output control circuit 73 enables four memorymodules 23, each normally associated with a separate PE, to beassociated with one of the PE's in response to the conditions of theADRS (11:10) high order address signals from the control microword.Thus, in one embodiment, in which the memory modules 23 each have on theorder of 1 Kb (1024 bits), when either of the ADRS (11) or ADRS (10)high order address bits are asserted, one PE has an effective memory of4 Kb (4096 bits), and the other PE's, which are normally associated withthe PE's whose memory modules 23 are available to the one PE, areeffectively disabled. This may be useful in several situations,including cases requiring more storage than is normally available to onePE. In addition, this may be useful in cases in which operations areperformed in each PE on data in parallel, and then operations areperformed serially by one PE using the data stored in all of the memorymodules. Making the data in the memory modules directly available to thePE performing the operations serially avoids requiring the PE's to usemessage transfers to transfer the data to that PE.

The data from the memory data output control circuit 73 is stored in aread data latch 74 and coupled by the latch to transceiver 72. Thetransceiver 72 in turn couples the data onto data bus 58. The data bus58, in turn, couples the data to one input terminal of a multiplexer 75.If an NR neighbor signal from the control microword is not asserted, themultiplexer 75 couples the read data from the data bus 58 onto a bus 77to the input terminal of Boolean logic circuit 54.

A multiplexer 76 selects a data signal from one of the neighboring PE's,in response to the condition of D(1:0) direction signals from thecontrol microword. If the NR neighbor signal from the control microwordis asserted, multiplexer 75 couples the signal selected by multiplexer76 onto the low-order line of bus 77, and transmits negated signals ontothe remaining lines of the bus, thereby facilitating the data receptionportion of the nearest-neighbor message transfer mechanism describedabove in connection with FIG. 3.

Data is coupled to the memory modules 23 for storage as follows. Datafrom the data bus 58 is coupled through transceiver 72 to the data inputterminals DI of the associated memory module 23. Depending on thecondition of the ADRS (11:10) signals, the memory data output controlcircuit 73 may instead couple the data from the transceiver 72 to amemory module 23 normally associated with another PE for storagetherein.

The data transmission portion of the nearest-neighbor message transfermechanism, which was described above in connection with FIG. 3, isperformed as follows. The DATA (0) signal from data bus 58 is connectedto one terminal of a multiplexer 80. If neither the ROW signal nor theCOL column signal from the control microword is asserted, themultiplexer 80 couples the DATA (0) signal to the input terminal of ademultiplexer 81. If the NR neighbor mode signal from the controlmicroword is asserted, the demultiplexer 81 couples the DATA (0) signalto the PE determined by the D(1:0) direction signals.

If, on the other hand, either the ROW or the COL column signal from thecontrol microword is asserted, the multiplexer 80 couples the outputsignal from an OR gate 82 to the demultiplexer 81. The OR gate 82 isenabled by the DATA (0) signal from data bus 58 or the output ofmultiplexer 76. The signal transferred to the neighboring PE thusrepresents the logical OR of the signal from a neighboring PE receivedduring the nearest neighbor transfer and the data signal on thelow-order line of data bus 58. By this mechanism, the row and columndata comparisons may be easily facilitated, which is useful inconnection with finding minimum and maximum values along a row or columnof PE's If both the ROW and COL signals are asserted at once, thecomparisons may be facilitated with respect to all of the PE's in thearray 11 at one time.

Also depicted on FIG. 4 is circuitry associated with the PE chip'srouter control circuit 33 for enabling the PE transfer messages over therouting network. As was noted above in connection with FIG. 2, the PE'son a chip are iteratively enabled to transfer messages over the routingnetwork. Each PE receives an RTR OUT EN IN router out enable in signalwhich, when asserted, enables the PE to transmit message data signalsover a common bus line 90 as RTR DATA OUT router data out signals. Afterthe message is acknowledged, the PE generates an RTR OUT EN OUT routerout enable out signal which is coupled to the next PE in line as thatPE's RTR OUT EN IN router out enable in signal.

Specifically, when the TR transmit request flag is set, if anacknowledge flip-flop 91, representing the ACK acknowledge flag 26, isclear, and so an MSG ACK message acknowledge signal is negated. Inresponse, an inverter 89 enables a NAND gate 92 to couple a low signalto one input of a NOR gate 93. When the PE's RTR OUT EN IN router outenable in signal is asserted, an inverter 94 couples a second low inputsignal to the other input terminal of NOR gate 93, thereby enabling theNOR gate to assert the TR EN transmit enable signal. The TR EN signal,in turn, controls the output enable terminal of a driver 95. When the TREN transmit enable signal is asserted, driver 95 couples the DATA (0)data signal from data bus 58 onto the line 90 as the RTR DATA OUT routerdata out signal.

The low signal from NAND gate 92 is complemented by an inverter 96 whichenergizes an OR gate 97, which, in turn, energizes the D data inputterminal of acknowledge flip-flop 91. At the end of the messagetransfer, an ACK DEL delayed acknowledgement signal is asserted over acommon line 100 to all of the PE's, the flip-flop 91 is set, therebyasserting the MSG ACK message acknowledge signal. In response, theinverter 89 disables NAND gate 92, which then generates a high outputsignal. Inverter 96 then couples a low signal to OR gate 97, but thesecond input signal to the OR gate, which comprises the now asserted MSGACK message acknowledge signal, maintains the acknowledgement flip-flopset if the ACK DEL delayed acknowledgement signal is later asserted.

The high output signal from NAND gate 92 also disables NOR gate 93 andenables an AND gate 101. The disabled NOR gate 93 negates the TR ENtransmit enable signal, in turn turning off driver 95. The enabled ANDgate 101 transmits the asserted RTR OUT EN OUT router out enable outsignal to the next PE.

It will be appreciated that if the TR transmit request flag 24 is notset, indicating that the PE is not enabled to transmit a message overrouting network 30 (FIG. 2), the TR transmit request flag couples a lowsignal to the NAND gate 92. Thus, the output signal from NAND gate 92will be high, disabling the NOR gate 93 and enabling AND gate 101 toassert the RTR OUT EN OUT router out enable out signal.

After all of the messages are transferred during a major transfer cycle,all of the acknowledgement flip-flops are reset by the assertion of anRST ACK reset acknowledgement signal from the control microword fromarray control unit 11.

In one embodiment, the RTR OUT EN IN router out enable in signal isalways asserted if the PE is the first PE in the hierarchy on a PE chipof the chip pair which transmits first, as described above in connectionwith FIG. 2. Accordingly, in that embodiment, the PE can begintransmitting a message immediately after its TR transmit request flag 24is set.

It will be appreciated that the assertion of the RTR OUT EN IN routerout enable in signal need not move only down the PE hierarchy on the PEchip. In particular, if a TR request flag 24 of a PE 22 is not set whenits RTR OUT EN IN router out enable in signal is asserted, it willimmediately assert its RTR OUT EN OUT router out enable out signal toenable the next PE in the hierarchy. However, if, during the majormessage transfer cycle, the PE receives a message which causes its TRtransmit request flag 24 to be set, since (a) its ACK acknowledge flaghas not been set, and (b) its RTR OUT EN IN router out enable in isstill asserted, AND gate 101 will be disabled so as to negate the RTROUT EN OUT router out enable out signal to the succeeding PE's in thehierarchy, and the driver 95 will immediately be enabled to couple theRD LTH (0) signal onto output line 90 as the RTR DATA OUT router dataout signal. This mechanism facilitates the "spreading activation" ofmessages described above.

B. Memory Interconnection

As has been noted, the array 11 can operate in either a 1 Kb mode inwhich each PE has an associated 1 Kb memory module 23 (FIGS. 2 and 4),or a 4 Kb mode. In the 4 Kb mode, the PE's are divided into groups offour PE's with one PE having access to its associated 1 Kb memory module23 as well as to the 1 Kb memory modules that are normally (that is, in1 Kb mode) associated with the three other PE's. This allows operationsto proceed in parallel by all of the PE's, and then permits the results,which are contained in all of the memory modules 23 to be accessed byone PE for future processing, without requiring the results to betransferred by way of messages. FIGS. 5A and 5B depict circuits forfacilitating this operation.

With reference to FIG. 5A, a group of four processors 22(0) through22(3) are connected to four memory modules 23(0) through 23(3) throughmemory control circuit 73. A multiplexer/demultiplexer 110 selectivelycouples data signals between all of memory modules 23(0) through 23(3)and PE 22(0), and a plurality of transceivers 111 through 113 eachcouple data signals between one memory module 23(1) through 23(3) andone associated PE 22(1) through 22(3). The ADRS (9:0) low order addresssignals from the control microword are coupled to all of the memorymodules 23(0) through 23(3) in parallel to select one location in eachof the memory modules, and ADRS (11:10) high order address signals fromthe control microword control the memory control circuit 73.

If the ADRS (11:10) signals are both negated, the array 11 (FIG. 1)operates in the 1 Kb mode. In that mode multiplexer/demultiplexer 110transfers data signals between the PE 22(0) and memory module 23(0). Thenegated ADRS (11) and ADRS (10) signals energize a NOR gate 114, which,in turn, energizes output enable terminals on each of the transceivers111 through 113. This enables the transceivers to couple data signalsbetween memory modules 23(1) through 23(3) and respective processors22(1) through 22(3).

If, on the other hand, either of the ADRS (11) or ADRS (10) signals areasserted, the array 11 operates in 4 Kb mode. In that case, NOR gate 114disables the transceivers, which enables them to transmit data signalsto the processors 22(1) through 22(3) having zero values. The ADRS(11:10) address signals enable the multiplexer/demultiplexer to transferdata signals between one of the memory modules 22(0) through 22(3), asdetermined by the binary value of the encoded ADRS (11:10) addresssignals, and PE 22(0).

FIG. 5B depicts a schematic diagram illustrating one embodiment of acircuit of the portion of the memory control circuit 73 between memorymodule 23(3) and PE 22(3), and showing the circuit for coupling datasignals to PE 22(0). When a location, as identified by the ADRS (9:0)address signals from the control microword, is being read, memory module23(3) transmits four data bit signals, identified as DAT(0) throughDAT(3) on respective lines 120(0) through 120(3). The DAT(0) data signalcorresponds to the contents of the bit location identified by the ADRS(9:0) address signals, and the other DAT(1) through DAT(3) signalscorrespond to the contents of the bit locations having the respectivenext sequential bit address locations.

If the array 11 is not in the 4 Kb mode, an OR gate 121 will bede-energized so as to negate a 4K MODE signal, and an inverter 122 willgenerate an asserted NOT 4K MODE signal. The negated 4K MODE signalturns on four pass transistors 123(0) through 123(3), which couples therespective DAT(0) through DAT(3) data signals to lines 130(0) through130(3) as DAT (0)-(3) TO PE (3) signals for storage in read latch 74,which is represented by four latches 74(0) through 74(3). The latchesstore the respective signals and couple them to the PE 22(3) as LAT DAT(0)-(3) TO PE (3) signals.

If the array is in the 4 Kb mode, the ADRS(11:10) address signals enablethe OR gate 121 to assert the 4K MODE signal and negate the NOT 4K MODEsignal. The negated NOT 4K MODE signal de-energizes transistors 123(0)through 123(3), thereby blocking the transmission of the DAT(0) throughDAT(3) signals to lines 130(0) through 131(3), and the asserted 4K MODEsignal energizes transistors 131(0) through 131(3) to ground the lines130(0) through 130(3) and negate all of the DAT (0)-(3) TO PE (3)signals. As a result all of the latches 74(0) through 74(3) are reset,resulting in negated LAT DAT (0)-(3) TO PE (3) data signals coupled toPE 22(3).

In addition, if both ADRS(11:10) address signals are asserted,indicating the data signals from memory module 23(3) are to be coupledto PE 22(0), a demultiplexer 124 energizes a line 125(3) to assert a MEMDAT (3) memory data signal, which energizes pass transistors 126(0)through 126(3). Energized pass transistors 126(0) through 126(3) couplethe DAT(0) through DAT(3) data signals o lines 120(0) through 120(3)onto lines 127(0) through 127(3) as DAT (0)-(3) TO PE (0) signals, whichare coupled to PE 22(0).

Data may be written from the PE 22(3) to memory module 23(3) if the NOT4K MODE signal is asserted when the WRT EN (PE3) write enable signal isasserted by router control circuit 33 (FIG. 4) The WRT EN (PE3) signalenables AND gates 132(0) through 132(3), which couple the DAT (0)-(3)FRM PE(3) data signals from the PE(3) to lines 130(0) through 130(3),respectively. The asserted NOT 4K MODE signal turns on the transistors123(0) through 123(3), enabling them to couple the DAT (0)-(3) FRM PE(3)data signals onto lines 120(0) through 120(3), after which they arestored in memory module 23(3).

The memory control circuits 73 associated with processors 22(1) and22(2) are similar to that depicted in FIG. 5B except that transistors126(0) through 126(3) are connected to lines 125(1) and 125(2),respectively, and controlled by the MEM DAT (1) and MEM DAT (2) signalsfrom demultiplexer 124.

FIG. 5C depicts the circuit of the memory control circuit 73 associatedwith PE 22(0). It can be seen that the memory control circuit 73depicted in FIG. 5C is similar to the circuit depicted in FIG. 5B,except that there are no transistors 123(0) through 123(3) or 131(0)through 131(3); instead, the lines 130(0) through 130(3) are connecteddirectly to lines 127(0) through 127(3), respectively, and the DAT(0)-(3) TO PE (0) data signals control the latches 74(0) through 74(3)directly. Furthermore, transistors 126(0) through 126(3) are controlledby the MEM DAT (0) signal from demultiplexer 124 to couple the DAT(0)through DAT(3) signals from memory module 23(0) directly onto the lines127(0) through 127(3) when the MEM DAT (0) signal is asserted. The MEMDAT (0) signal is asserted by demultiplexer 124 when the ADRS (11) andADRS (10) signals are both negated, that is, when the array 11 is not inthe 4 Kb mode.

C. Status Signal Generation

As was noted above in the discussion regarding FIGS. 2 and 4, the arraycontrol unit 10 (FIG. 1) uses a DONE status signal transmitted overstatus bus 15 to determine if various operations have been completed bythe PE's 21 in array 11 and to control further processing. The DONEstatus signal is asserted, for example, during message transfer overrouting network 30 (FIG. 2) when all of the PE's which have messages tosend (that is, whose TR transmit request flags 24) have receivedacknowledgements that their messages have been received during a majormessage transfer cycle. The DONE status signal is also asserted duringnormalization during floating point arithmetic when the fraction portionof the floating point numbers in all of the PE's have been normalized.

In both the transfer of messages and normalization, until the DONEsignal is asserted, the array control unit 10 repetitively transmitssequences of control microwords which enable the array 11 to engage in amessage transfer operation or in a normalization operation. Whether ornot a particular PE actually engages in the operation depends on thestate of various ones of the PE's flags. As has been described, whetheror not a PE engages in a message transfer depends on the state of the TRtransmit request flag 24, the ACK acknowledge flag 26 and the PE's RTROUT EN IN router out enable in signal (see FIG. 4). Furthermore, whetheror not a PE engages in a normalization operation depends on thecondition of the PE's M flag 62 (FIG. 4).

FIG. 6 depicts circuitry within a PE chip 20 for generating a DONE OUTsignal which is used by array 11 (FIG. 1) in connection with generatingthe DONE status signal. With reference to FIG. 6, each PE 22(0) through22(N) on the PE chip 20 has extending therefrom a wire 150(0) through150(N) which carries signals M(0) through M(N). The asserted or negatedcondition of each signal M(0) through M(N) corresponds to the set orcleared condition of the PE's M flag 62 (FIG. 4). Thus, if a PE's M flag62 is set, the PE's M (x) signal ("x" a value from zero to N) isasserted, otherwise the M(x) signal is negated.

All of wires 150(0) through 150(N) are connected to a common wire 151 ina wired-OR configuration. Wire 151 carries an M COMP composite "M"signal whose asserted or negated condition depends on the conditions ofthe M(0) through M(N) signals from all of the PE's on the chip. Thus, ifthe M flag 62 (FIG. 4) of any of processors 22(0) through 22(N) is set,the PE's corresponding M(x) signal is asserted, and the M COMP signal isasserted. If the M flag 62 of all of processors 22(0) through 22(N) iscleared, all of the M(0) through M(N) signals will be negated, and the MCOMP signal will be negated. As described above in connection with FIG.4, when a PE has completed the normalization operation, its M flag 62will be cleared; thus, when the M COMP composite M signal is negated,all of the PE's on the chip will have completed the normalizationoperation.

As also described above in connection with FIG. 4, as each PE 22(x)receives acknowledgement of a message transmitted through routingnetwork 30 (FIG. 2), it asserts its RTR OUT EN OUT (x) router out enableout signal, which enables the next PE 22(x+1) on the chip to transmitits message. If the PE 22(x) is not enabled to transmit a message, thatis, if its TR transmit request flag 24 is cleared, when it receives theRTR OUT EN IN router out enable in signal from the preceeding PE 22(x-1)(which corresponds to that PE's RTR OUT EN OUT router out enable outsignal), it asserts its RTR OUT EN OUT router out enable out signalimmediately. Thus, when the last PE 22(N) asserts its RTR OUT EN OUT (N)router out enable out [PE 22(N)] signal, all of the PE's on the chipwhich were enabled to transmit messages over the routing network havetransmitted the messages and have received acknowledgements that thosemessages have been received.

With reference to FIG. 6, the inverter 152 generates the complement ofthe RTR OUT EN OUT (N) router out enable out [PE 22(N)] signal from PE22(N) places it on a line 153. Thus, the signal on line 53 is at a lowvoltage level when the RTR OUT EN OUT (N) router out enable out [PE22(N)] signal is asserted (high voltage level). Lines 151 and 153 areconnected to two input terminals of a multiplexer 154, which iscontrolled by a DN done select signal from the control microword. The DNdone select signal enables the multiplexer to couple the signal on oneof lines 151 or 153 to its output terminal as a DONE X signal, which, inturn, is coupled to one input terminal of an OR gate 155. If the signalon the one of lines 151 or 153, as selected by the DN done selectsignal, is asserted, the DONE X signal is also asserted, otherwise, theDONE X signal is negated. If the DONE X signal is asserted, the OR gate155 also asserts the DONE OUT signal.

The OR gate 155 has a second input terminal which receives a DONE PREVdone previous signal from an OR gate 156. OR gate 156, in turn receivesas input signals DONE IN A through DONE IN D, each of which correspondsto a DONE OUT signal from another PE chip. The DONE OUT signal from thePE chip 20 depicted in FIG. 6 may also be directed as an input signal toan OR gate corresponding to OR gate 156 on another PE chip. Theresulting connection, an "OR" tree, results in one DONE OUT signal fromone PE which is coupled to array control unit as the DONE signal. Thus,when the DONE signal is in a low voltage condition, depending on thecondition of the DN done select signal, either all of the M COMPcomposite M signals in all of the PE chips are negated, indicating thatall of the M flags 62 are cleared, or all of the RTR OUT EN OUT (N)router out enable out signals are asserted, indicating that all messageshave been transmitted and acknowledged. Thus, when the DONE signal is ina low voltage condition, the array 11 has completed the major messagetransfer cycle, or the operation (such as floating point numbernormalization) which was governed by the M flags 62 in the PE's.

D. Shift Register

As described above in connection with FIG. 4, the A shift register 52(FIG. 4) can shift not only toward its S/O output terminal, but alsotoward its S/I input terminal. This capability permits the contents ofthe shift register to be easily shifted prior to such operations asfloating point addition and subtraction, which requires the fractionportion of the floating point number having a lower valued exponent tobe shifted so as to represent a lower value.

FIG. 7 depicts a schematic circuit of a portion of the A shift register52. The A shift register 52 includes, in one embodiment, eight cells160(0) through 160(7), each of which includes four stages. Each stagestores one data bit. Cell 160(0) contains the least significant databits (indicated by the legend "LSB" in the Figure) and the succeedingcells 161(1), 161(2), and so forth, contains successively moresignficant data bits, so that, depending on the A SHFT SZ shift registersize signals from the control microword (see FIG. 4) cell 160(7) maycontain the most significant data bits (as indicated by the legend "MSB"in the Figure). Data is shifted into A shift register 52 through cell160(0). The cell from which data bits are shifted out depends on thecondition of the A SHFT SZ A shift register shift size signals from thecontrol microword (see FIG. 4).

Since the circuits of the cells are similar, the circuit of only cell160(4) will be described in detail. The cell 160(4) includes four stages161(0) through 161(3) each of which includes a pair of inverters,including input inverters 162(0) through 162(3) and output inverters163(0) through 163(3). Stage 161(0) stores the low-order data bit of thedata in the cell, and stage 161(3) stores the high-order data bit of thedata in the cell.

The pair of inverters 162(n) and 163(n) ("n" is an integer from 0 to 3)in each stage are connected through control transistors 180(n) and181(n) to form a flip-flop which actually stores the data bit. In eachstage, the output of inverter 162(n) is coupled through a controltransistor 180(n), which is normally held on by an asserted PH 1 controlsignal, to the input of inverter 163(n). The output of inverter 163(n)is coupled through control transistor 181(n) to a node 182(n), to whichthe input of inverter 162(n) is also connected. The transistor 181(n),which is normally held on by an asserted HOLD control signal, controlsthe feedback from the output of inverter 163(n) to the input of inverter162(n) which enables the inverters to latch the input at node 182(n) ina known manner.

During shift operations, the HOLD signal is negated, turning offtransistor 181(n) and breaking the feedback path. In addition, the PH 1timing is also negated, turning off transistor 180(n). Transistor180(n), when off, isolates the output inverter 163(n) from the inputinverter 162(n). This allows the output inverter 163(n) of the stage totransmit a signal representing the stored data bit at the same time thatthe input inverter 162(n) of the stage receives a signal representing asignal representing a data bit from another stage as described below.

Cell 160(4) receives four bits of data, comprising signals DATA IN (0)through DATA IN (3), from cell 160(3) on respective lines 164(0) through164(3). Cell 160(4) transmits four bits of data, comprising signals DATAOUT (0) through DATA OUT (3), on respective lines 165(0) through 165(3),to stage 160(5). In addition, if the stage 160(4) is one from which datasignals are transmitted to multiplexer 56 (FIG. 4), each stage includesa pass transistor 170(0) through 171(3), connected to lines 165(0)through 165(3) and controlled by a CELL 4 OUT signal, in the A SHFT SZ Ashift register 52 size signals. When the CELL 4 OUT signal is asserted,the transistors 170(0) through 170(3) are enabled to couple the DATA OUT(0) through DATA OUT (3) signals to the A shift register input terminalof multiplexer 56.

The direction and amount of shift in A shift register 52 are controlledby the SHFT DIR shift direction signal and the NM nibble mode signal,both of which are provided by the control microword from the arraycontrol unit 10 and by an EN SHFT enable shift timing signal, inaddition to the timing control signals HOLD and PH 1. A shift register52 generates three additional control signals in response to the SHFTDIR and NM signals. Specifically, A shift register 52 includes an ANDgate 171 which receives the SHFT DIR and NM signals and generates an NSnibble shift signal, which, when, asserted, enables one input of an ANDgate 176. If the NS signal is asserted, when the EN SHFT enable shifttiming signal is then asserted, AND gate 176 asserts an NSU nibble shiftup signal. As will be described below, when the NSU signal is asserted,the A shift register 52 is conditioned to shift the data bits up anibble (4 bits) at a time, that is, to the cell 160 which stores thenext most significant data bits. Thus, when NSU nibble shift up signalis asserted, the data bits are conditioned to be shifted from cell tocell, and are stored in the same stages in each cell. The data bitsshifted into cell 160(0) are provided by full adder 51.

A second AND gate 172 generates a BS bit shift signal in response to thecomplement of the NM nibble mode signal, as inverted in inverter 173,and the SHFT DIR shift direction signal. Thus, when the NM nibble modesignal is negated and the SHFT DIR shift direction signal is asserted,the BS bit shift signal from AND gate 172 will be asserted. When the ENSHFT enable shift timing signal is asserted, AND gate 177 asserts theBSU bit shift up signal. Since, however, the NM signal is negated, theNS nibble shift signal from AND gate 171 will be negated and the NSUnibble shift up signal from AND gate 176 will remain negated when the ENSHFT enable shift signal is asserted. However, as will be describedbelow, when the BSU bit shift up signal is asserted, the A shiftregister 52 is conditioned to shift the data bits up a stage at a time.The data bit in the most significant stage in each cell is shifted intothe least significant stage of the next more significant cell. The databit shifted into the least significant stage [corresponding to stage161(0)] is provided by the ADD OUT (0) signal from full adder 51 (seeFIG. 4).

Another AND gate 174 generates a DOWN signal in response to thecomplement of the NM nibble mode signal and the complement of the SHFTDIR shift direction signal, as inverted in inverter 175. When the ENSHFT enable shift timing signal is next asserted, an AND gate 178enabled by the DOWN and EN SHFT enable shift signal asserts the SD shiftdown signal. Thus, when both the SHFT DIR shift direction signal and theNM nibble mode signal are negated, the DOWN signal is asserted. Inaddition, with the SHFT DIR and NM signals both negated, both the NSUand BSU nibble and bit shift up signals from AND gates 176 and 177 arenegated. As will be described below, when the SD shift down signal isasserted, the A shift register 52 is conditioned to shift the data bitsdown a stage at a time. The data bit in the least significant stage ineach cell, except for the most significant stage in cell 160(7), isshifted into the the most significant stage of the next lessersignificant cell. A data bit having the value zero is stored in the mostsignificant stage [corresponding to stage 161(3)] of the mostsignificant cell 160(7).

Each stage 161(0) through 161(3) further includes three transistors,namely transistors 183(n), 184(n) and 185(n) each enabled by one of theBSU bit shift up, NSU nibble shift up, and SD shift down controlsignals, respectively. When a control signal is asserted, the enabledtransistor couples a signal representing the state of a data bit fromanother stage to the input node 182(n) of its respective stage.

In response to the assertion of the BSU bit shift up signal, withreference first to stage 161(0), the transistor 183(0) is turned on,coupling the DATA IN (3) data signal on line 164(3) to node 182(0). TheDATA IN (3) data signal corresponds to the data bit from the mostsignificant stage of cell 160(3). Further, in response to the BSU bitshift up signal, transistors 183(1) through 183(3) are turned on,coupling the DATA OUT (0) through DATA OUT (2) signals to nodes 182(1)through 182(3). As a result, the data bits in each of the stages, namelythe most significant stage of cell 160(3) and the stages 161(0) through161(2) are shifted up one stage and are stored in stages 161(0) through161(3) of cell 160(4).

In response to the assertion of the NSU nibble shift up signal,transistors 184(0) through 184(3) are turned on, coupling the DATA IN(0) through DATA IN (3) data signals on lines 164(0) through 164(3),respectively to nodes 182(0) through 182(3). The DATA IN (0) throughDATA IN (3) data signals correspond to the data bits from the respectivestages of cell 160(3). As a result, the data bits in each of the stagesof cell 160(3) are shifted up one cell (four stages) and are stored instages 161(0) through 161(3) of cell 160(4).

In response to the assertion of the SD shift down signal, transistors185(0) through 185(2) are turned on, coupling the DATA OUT (1) throughDATA OUT (3) data signals on lines 165(1) through 165(3), respectivelyto nodes 182(0) through 182(2) for storage in stages 161(0) through161(2). In addition, transistor 185(3) is turned on to couple the DATAOUT (0) [CELL 160(5)] data signal from the least significant stage incell 160(5) [that is, the stage corresponding to stage 161(0)] to node182(3) for storage in stage 161(3). A transistor in the most significantstage of cell 160(3) corresponding to transistor 185(3) receives theDATA OUT (0) data out signal from stage 161(0) for storage therein. As aresult, the data bits in each of the stages are shifted down one stage.

IV. Routing Network 30

A. General Description

As described above with reference to FIG. 2, the routing network 30 isused to transmit messages between PE's which are typically not likely tobe on the same PE chip. A general organizational diagram of oneembodiment of a routing network constructed in accordance with theinvention is depicted in FIG. 8A. With reference to FIG. 8A, the routingnetwork 30 comprises a plurality of identical switching chips (eachswitching chip is depicted as a rectangle in the Figure) organized inthree switching stages 201, 202 and 203. The internal switching circuitsof chips 200(n) will be described below in connection with FIGS. 9 and10A through 10C.

Before proceeding further, it would be helpful to describe the format ofa message transmitted through routing network 30. With reference to FIG.8B, a message begins with a header 204 of twenty-three bits, namely bits(0) through (22), which identify the intended recipient PE, followed bythe message data bits beginning with bit (23). The header includes threerouter control fields, identified by reference numerals 205-207, each ofwhich controls the switching through one of the switching stages201-203, respectively. Each switching stage 201-203 retires one routercontrol field, that is, it does not pass the bits in that field on tothe next switching stage or to the recepient PE chips.

Each router control field begins with a protocol bit P (which isasserted at a low voltage level) which indicates that message bitsfollow. If the protocol bit P is not received at an input terminal, theswitching chips ignore succeeding signals at that input terminal duringthe minor message transfer cycle. The four bits following each protocolbit are RTR ADRS router address bits which are used by the router toestablish a switching path through the stage as described below inconnection with FIGS. 9 and 10A through 10C. If a switching path isestablished through all of the switching stages, a protocol bit P (bit15) is transmitted to the PE chip pair (see FIG. 2) connected to theoutput line from stage 203. In response, during bit 16, the PE chipsgenerate an ACK acknowledgement signal which is transmitted over theswitching path established through the routing network 30. At this time,the switching chips condition the switching path to receive a signalfrom the recipient PE chip pair and couple it to the PE chip 20 which isoriginating the message. The PE chip 20 originating the message uses theACK acknowledgement signal to set the transmitting PE's ACKacknowledgement flag 26 (FIGS. 2 and 4).

After the transfer of the ACK acknowledgement signal, a six-bit PROC IDprocessor identification is transmitted over the switching path. Thefirst bit identifies the PE chip 20 in the chip pair which contains thePE to receive the message, and the last five bits identify the PE on theidentified chip. The next bits are the message bits which are coupled bythe PE chip's router control circuit 33 (FIGS. 2 and 4) to the PE.

With reference again to FIG. 8A, the first two switching stages 201 and202 are divided into four switching groups, one of which is depicted inthe Figure. The last switching stage 203 operates as a crossbar switchto switch the outputs from the four switching groups to the PE chips.Since the four switching groups are identical, only the one depicted inthe Figure will be described in detail. That switching group includessixteen switching chips, identified by reference numerals 201(0) through201(15) in switching stage 201, and an additional sixteen switchingchips, identified by reference numerals 202(0) through 202(15) inswitching stag 202.

Each switching chip 201(n) and 202(n) ("n" is an integer) has sixty-fourinput terminals, generally identified by reference numeral 210, andsixty four output terminals, generally identified by reference numeral211. The sixty-four output terminals are grouped into sixteen outputgroups each having four output terminals. Each of the sixteen outputgroups is identified by one encoding of the four RTR ADRS router addressbits in the portion of the header 204 (FIG. 8B) that is retired by theswitching chip. On receiving at an input terminal RTR ADRS routeraddress bits having a particular encoding, the switching chip attemptsto open a switching path from that input terminal to one of the fouroutput terminals in the output group identified by that encoding If RTRADRS router address bits received at four or fewer input terminals ofthe switching chip identify a particular output group, switching pathscan be established to the output group from all of the input terminals.If, on the other hand RTR ADRS router address bits identifying aparticular output group are received at more than four input terminals,at most four switching paths can be established to the output group andthe other messages, for which switching paths cannot be established, areblocked.

The output terminals of the switching chips 201(0) through 201(15)included in the first switching stage 201 are connected to the inputterminals of the switching chips 202(0) through 202(15) of the secondswitching stage as follows. All of the output terminals of the chips201(0) through 201(15) in groups associated with the "0000" encoding ofthe RT ADRS router address retired by switching chips 201(0) through201(15) are connected to the input terminals of switching chip 202(0).This is depicted in the Figure by the wires identified by the legends"4×0000" extending from the right sides of switching chips 201(0)through 201(15) to the left side of switching chip 202(0). (In thelegend "4×0000" in FIG. 8A, the first number, "4", indicates that theassociated line includes 4 connecting wires, and the second number"0000" indicates that the wires are associated with the "0000" encodingof the first four RTR ADDRS router address signals, which are retired bythe switching stage 201.)

Similarly, all of the output terminals of the chips 201(0) through201(15) in groups associated with the "0001" encoding of the RTR ADRSrouter address retired by switching chips 201(0) through 201(15) areconnected to the input terminals of switching chip 202(1), as depictedin FIG. 8A by the wires identified by the legends "4 x 0001" extendingfrom the right sides of switching chips 201(0) through 201(15) to theleft side of switching chip 202(1). This interconnection pattern betweenthe switching chips in the first two stages 201 and 202 is repeated forall of the other switching chips 202(2) through 202(15); as depicted inthe Figure, the input terminals of switching chip 202(15) are connectedto the output terminals of switching chips 201(0) through 201(15) thatare in groups associated with the "1111" encoding of the RTR ADRS routeraddress retired by the switching chips 201(0) through 201(15). Thus, theoutput terminal groups in each of the switching chips 201(0) through201(15) are connected to diverse ones of the switching chips 202(0)through 202(15) in the second switching stage 202, with the connectiondepending on the RTR ADRS router address that is retired by theswitching chips in the first stage.

As described above, the sixty-four output terminals of each of theswitching chips 202(0) through 202(15) in the second switching stage 202are also divided into sixteen groups each having four terminals. Likeswitching chips 201(0) through 201(15), each output terminal group ofswitching chips 202(0) through 202(15) is associated with one encodingof the four RTR ADRS router address signals retired by those chips. Theoutput terminal groups of switching chip 202(0), for example, connect towires having the legends "4×0000 0001" through "4×0000 1111", with "4"indicating the number of wires (with each wire being connected to oneoutput terminal), the first block of four digits "0000" indicating theRTR ADRS router address bits retired by the first switching stage 201,and the second block of four digits "0000" and "1111" indicating the RTRADRS router address bits that are retired by the switching chips 202(0)through 202(15). The wires attached to the output terminal groups ofswitching chips 202(1) through 202(15) are identified by similarlegends.

As described above, switching stages 201 and 202 are divided into fourswitching groups (one group is depicted in the Figure) the outputs ofwhich are switched by crossbar stage 203. The other three switchinggroups of switching stages 201 and 202 include chips corresponding toswitching chip 202(0) having output terminal groups associated with theRTR ADRS router address bits "0000 0000" through "0000 1111", chipscorrepsonding to switching chip 202(1) having output terminal groupsassociated with the RTR ADRS router address bits "0001 0000" through"0001 1111", and so on.

The crossbar switching stage 203 includes sixteen crossbar switchingblocks of sixteen crossbar switching chips 203(0) through 203(255) [onlyswitching chips 203(0) through 203(3) are shown in the Figure], witheach crossbar switching block switching the output signals from outputterminal groups in the corresponding switching chips in each of theswitching groups. Each of the sixteen crossbar switching chips in eachcrossbar switching block, in turn, switches the outputs fromcorresponding terminal groups in corresponding stage 202 switching chipsfrom all of the four groups. Thus, crossbar switching chip 203(0) isconnected to, and switches the outputs from, the "4×0000 0000" outputterminal groups from switching chip 202(0) and corresponding switchingchips in each of the other switching groups.

Similarly, crossbar switching chip 203(1) is connected to, and switchesthe outputs from, the "4×0000 0001" terminal groups from switching chip202(0) and corresponding switching chips in each of the other switchinggroups. In addition, the last crossbar switching chip in the crossbarswitching block associated with switching chip 202(0) (and correspondingswitching chips in the other switching groups), namely crossbarswitching chip 203(15), is connected to, and switches the outputs from,the "4×0000 1111" terminal groups from switching chip 202(0) andcorresponding switching chips in each of the other switching groups.

The crossbar switching chips 203(16) through 203(255) in each of theother crossbar switching blocks is similarly connected to the otherswitching chips 202(1) through 202(15), and corresponding switchingchips in the other switching groups.

Each crossbar switching chip operates as a crossbar switch. That is, thecrossbar switching chip associates each of its sixteen output terminalswith one encoding of the final four RTR ADRS router address bits whichit receives from the switching stage 202 switching chips. Thus, forexample, switching chip 203(0) has sixteen output terminals eachconnected to a wire having the legend "1×0000 0000 0000" through "1×00000000 1111". In this case, the leading "1" indicates that there is onewire, the first group of digits, namely the first group of "0000",corresponds to the RTR ADRS router address bits retired by switchingstage 201, the second group of digits, namely the second group of"0000", corresponds to the RTR ADRS router address bits retired byswitching stage 202, and the third group of digits, which range from"0000" through "1111", corresponds to the RTR ADRS router address bitsreceived and retired by the crossbar switching chip 203(0).

The output terminals of the other crossbar switching chips 203(1)through 203(255) have similar associations with the RTR ADRS routeraddress bits. It will be appreciated by those skilled in the art that:

(1) each of the sixteen output terminals of each crossbar switching chip203(0) through 203(255) is associated with one encoding of the thirdgroup of RTR ADRS router address bits (that is the third group of digitsassociated with each output line from the crossbar switching chipsdepicted in FIG. 8A,)

(2) each of the sixteen crossbar switching chips in each crossbarswitching block is associated with one encoding of the second group ofRTR ADRS router address bits (that is the second group of digitsassociated with each output line from the crossbar switching chipsdepicted in FIG. 8A,), and

(3) each of the sixteen crossbar switching blocks is associated with oneencoding of the first group of RTR ADRS router address bits (that is thefirst group of digits associated with each output line from the crossbarswitching chips depicted in FIG. 8A,).

Accordingly, the routing network depicted in FIG. 8A provides switchingpaths from the input terminals of the switching chips of stage 201 tothe output terminals of crossbar switching chips of crossbar stage 203for all possible encodings of the RTR ADRS router address bits.

The routing network 30 depicted in FIG. 8A has a number of advantages,in particularly relating to the association of multiple output terminalsfrom switching stages 201 and 202 into groups associated with the sameencoding of the RTR ADRS router address bits. This significantly reducesthe number of switching stages that are required, reducing the number ofswitching chips and wires interconnecting the chips. This, in turn,serves to enhance the reliability of the routing network, since thefailure rate is directly related to the number of chips and,particularly, wires interconnecting those chips.

In addition, by associating multiple wires in a terminal group with eachencoding of the RTR ADRS router address bits, if a wire, or itsassociated driving circuitry on the chip, is defective, the switchingchips will still be able to transmit over the remaining wires in theterminal group. The message traffic that can be accommodated will besomewhat reduced, but the messages can still be transferred through therouting network 30.

B. Switching Chip

The internal organization and circuitry within a switching chip 201(n)or 202(n) will now be described in connection with FIGS. 9 and 10Athrough 10C. As will be made clear in the following discussion, the samechip as described herein can also be used as a crossbar switching chipin the crossbar stage 203.

FIG. 9 depicts the general circuit organization of a portion of theswitching chip constructed in accordance with the invention. Withreference to FIG. 9, a switching chip includes a plurality of switchingcircuits each associated with one input terminal. One such switchingcircuit, designated by the reference numeral 210(0), is shown in FIG. 9.In one embodiment, a switching chip includes sixty-four switchingcircuits. All of the switching circuits are identical, and so only oneis depicted in the Figure. Each of the switching circuits is associatedwith one of the input terminals; switching circuit 210(0) is associatedwith the input (0) terminal, and generally switching circuit 210((u)("u" is an integer) is associated with the input (u) terminal.

Switching circuit 210(0) includes an input (0) circuit 211(0), sixteencoupling groups 212(0) through 212(15) [generally designated 212(v)] andsixteen output terminal groups 217(0) through 217(15) [generallydesignated 217(v)]. Each output terminal group includes four outputcircuits generally identified by the reference numeral 217(v)(w). Theinput (0) circuit 211(0), the circuit of which is described below inconnection with FIG. 10A, receives an IN(0) input signal from a PE chippair if the chip is in stage 201 (see FIG. 8A) or from a preceedingswitching stage if the chip is in stages 202 or 203. In response to theIN(0) input signal the input (0) circuit 211(0) transmits an ADRS(0)address signal onto a line 213(0) representing complement of the the RTRADRS router address bits to be retired by the chip and a DAT(0) datasignal onto a line 214(0) representing the remaining bits of themessage.

Each coupling group 212(v) includes an address decoding and controlcircuit 215(0) through 215(15) [generally designated by referencenumeral 215(v)] and a coupling circuit 216(0) through 216(15) [generally216(v)]. One circuit for a coupling group will be described in detailbelow in connection with FIGS. 10B-1 and 10B-2. Line 213(0) from input(0) circuit 211 couples the ADRS(0) address signal to the addresscircuit 215(0) in coupling group 212(0) and line 214(0) couples theDAT(0) data signal to all of the 216(v) coupling circuits in parallel.In response to the four RTR ADRS router address bits retired by theswitching chip, one of the address control circuits 215(v) in switchingcircuit 210(0) will assert an ADRS(v)EN(u) signal. For example, if theIN(0) signal received by input (0) circuit 211(0) has RTR ADRS routeraddres bits having the encoding "0000", address control circuit 215(0)will assert the ADRS(0)EN(0) signal.

Each coupling circuit includes four coupling modules generallydesignated by reference numeral 216(v)(w), with the (v) indexidentifying the coupling circuit and the (w) index identifying themodule within the circuit. For example, the coupling modules of couplingcircuit 216(0) depicted in FIG. 9 are designated by the referencenumerals 216(0)(0) through 216(0)(3), the coupling modules of couplingcircuit 216(1) being designated by the reference numerals 216(1)(0)through 216(1)(3), and so forth.

The coupling modules 216(v)(0) through 216(v)(3) in each switchingcircuit 210(0) through 210(63) are connected to one of sixty-four datalines 220(0)(0) through 220(15)(3) [generally designated by referencenumeral 220(v)(w)]. Data line 220(0)(0) is connected in parallel to allof the coupling modules 216(0)(0) in all of the switching circuits210(0) through 210(63), data line 220(0)(1) is connected in parallel toall of coupling modules 216(0)(1) in all of the switching circuits210(0) through 210(63) and so forth.

At any one time only one of the coupling modules connected to one dataline 220(v)(w) is enabled, as described below, to couple a data signal,corresponding to the DAT(u) data signal from its respective input (v)circuit 211(v), onto the data line 220 as a GRP(v)DAT(w) group datasignal. The output terminal group circuit 217(v) receives theGRP(v)DAT(w) group data signals relating to its terminal group andcouples them as the switching chip's output signals.

Eachcoupling module 216(v)(w) is enabled to couple the DAT(u) datasignal onto the respective data line 220(u)(v) in response to twoenabling signals. One enabling signal, the ADRS(v)EN(u) coupling groupenabling signal, generated by the address control circuit 215(v) inresponse to the decoding of the RTR ADRS router address bits from theADRS(u) signal.

The second signal which enables a coupling module 216(v)(w) in switchingcircuit 210(u) is a GRP(v)EN/(w)IN(w) enabling signal. For clarity inFIG. 9, only the GRP(0)EN(0)IN(0) through GRP(0)EN(3)IN(0) enablingsignals associated with output terminal group circuit 217(0) aredepicted. In an actual circuit, the other output terminal group circuits217(1) through 217(15) would also generate corresponding enablingsignals which would be coupled to coupling modules 216(n)(0) through216(n)(3) in coupling circuits 216(1) through 216(15).

Four daisy chain GRP(v)EN(w) enabling signals are originated by eachoutput terminal group circuit 217(v) used by the coupling modules216(v)(w) in the switching circuits 210(u). The GRP(v)EN(w) enablingsignals are normally asserted by the output terminal group circuits217(v). However, under some circumstances, such as, for example, if acommunications path to the next stage is disrupted due to a broken wireor the like, the output terminal group circuit 217(v) negates thecorresponding GRP(v)EN(w) enabling signal. Circuitry for detecting thepresence of a disrutped communication path is described below inconnection with FIGS. 10A and 10C.

The signals from the output terminal group circuits 217(v) are initiallycoupled to the coupling modules 216(v)(w) of the switching circuit210(0) as a GRP(v)EN(w)IN(0) enabling signal depicted on FIG. 9. If acoupling module 216(v)(w) in switching circuit 210(0) receives anasserted GRP(v)EN(w)IN(0) enabling signal, and is enabled by theADRS(u)EN(v) signal from the address control circuit, it transmits anegated GRP(v)EN(w)OUT(0) enabling signal to coupling module 216(u)(v)of switching circuit 210(1). In that case, the coupling module willcouple the DAT(0) signal onto the 220(u)(v) data line. On the otherhand, if a coupling module 216(u)(v) is not enabled by the ADRS(u)EN(v)signal from the address control circuit, it asserts a GRP(v)EN(w)OUT(0)enabling signal, which is coupled to coupling module 216(v)(w) ofswitching circuit 210(1). The coupling modules 216(v)(w) of that, andother switching circuits 210(2) through 210(63) operate similarly.

Thus the GRP(v)EN(w) enabling signals, originating from the outputterminal groups 217(v), are passed through the coupling modules216(v)(w) in a daisy-chained manner. If a coupling module 216(v)(w) isenabled by the associated address control circuit 215(v) and receivesthe GRP(v)EN(w) enabling signal in an asserted condition [that is, itsGRP(v)EN(w)IN(u) signal is asserted], the coupling module uses theassociated data line 220(v)(w) and blocks the GRP(v)EN(w) enablingsignal [that is, it transmits the GRP(v)EN(w)OUT(u) enabling signal tothe next switching circuit in a negated condition].

In any coupling group 216(v), only one coupling module 16(v)(w) will beenabled to couple the DAT(u) signal onto the associated data line220(v)(w). Thus, for example, if the leftmost coupling module 216(0)(0)(as depicted in FIG. 9) is enabled by the ADRS(0)EN(0) signal from theaddress control circuit 215(v), but is disabled by a negatedGRP(0)EN(0)IN(0) signal, it in turn enables the coupling module216(0)(1) to its immediate right. If that coupling module 216(0)(1), isenabled by an asserted GRP(0)EN(1)IN(0) signal, it will couple theDAT(0) signal onto the data line 221(0)(1). However, if theGRP(0)EN(1)IN(0) signal is negated, that coupling module 216(0)(1) willenable the coupling module 216(0)(2) to it its immediate right.Essentially, the ADRS(0)EN(0) enabling signal from address controlcircuit 215(0) is essentially daisy-chained through the coupling modules216(0)(0) through 216(0)(3).

For any received message, in any switching circuit 210(u) at most one ofthe ADRS(v)(EN(u) enabling signal from one of the address controlcircuits 215(v) will be asserted. That is, for any message only onecoupling circuit 216(v) in any switching circuit 210(u) will be enabledby an address control circuit 215(v). The circuit of an address controlcircuit 215(v) will be described below in connection with FIG. 10B-1. Inbrief, each address control circuit 215(v) includes an input terminal221(v) and an output terminal 222(v). Each address control circuit215(v) receives through its input terminal 221(v) an input signal whichis related to the ADRS(u) address signal, which in turn is related tothe RTR ADRS router address bits. The address control circuit 215(v)also includes a latch and an inverter. As long as the latch is reset,the address control circuit couples, through the inverter, thecomplement of the signal at its input terminal 221(v) to the outputterminal 222(v). If the latch is set, it remains set throughout themessage, and causes the inverter to be bypassed, thus enabling the truevalue of the signal at the input terminal 221(v) to be coupled to theoutput terminal 222(v).

As depicted in FIG. 9, the address control circuits 215(v) in aswitching circuit 210(u) are connected serially, so that each inputterminal 221(1) through 221(15) of address control circuits 215(1)through 215(15) is connected to the output terminal 222(0) through222(14), respectively, of the address control circuit 215(0) through215(14) to its left. Initially, all of the latches are reset.Accordingly, if the IN(0) signal carries an initial RTR ADRS routeraddress bit having the binary value "0", the ADRS(0) signal received byat input terminal 221(0) is asserted. As a result, the inverter in theaddress, control circuit 215(v) provides an negated output signal at itsoutput terminal 222(0), which is coupled to the input terminal 221(1) ofaddress control circuit 215(1).

Address control circuit 215(1), in turn receives the negated inputterminal through its input terminal 221(1) and its inverter couples anasserted output signal to the output terminal 222(1), which is, in turn,coupled to the input terminal 222(2) of address control circuit 215(2).The remaining address control circuits 215(v) of switching circuit210(0) operate similarly. Thus, when the address control circuit 215(15)receives an input signal through its input terminal 221(15), the inputsignals to address control circuits 215(v) having an even-numbered orzero index (v) will be asserted, and the input signals to the addresscontrol circuits having an odd-numbered index (v) will be negated.

At this point, the latches in the address control circuits 215(v) latchthe input signal. If the input signal to an address control circuit215(v) is asserted, the latch remains cleared, otherwise the latch isset. After a latch is set, it enables its address control circuit'sinverter to be by-passed, and so the signal received at it's inputterminal 221(v) will be coupled directly to the output terminal 222(v).If the latch is set, the address control circuit 215(v) is alsoinhibited from asserting the ADRS(v)EN(u) coupling circuit enablingsignal.

The input (0) circuit then transmits an ADRS(0) signal related to thesecond RTR ADRS router address bit to the input terminal 221(0) ofaddress control circuit 215(0). The address control circuits 215(v) ("v"an even number or zero) whose latches are cleared operate in the samemanner as described above and transmit a signal at their outputterminals 222(v) which is the complement of the signal at their inputterminals 221(v). At the same time, the address controls circuits 215(v)("v" an odd number) whose latches are set pass the signal which theyreceive at their input terminals 221(v) to their output terminals222(v). Thus, the input signals at the address control circuits 215(0)and others 215(v) whose indexes (v) are divisible by four asserted.Contrariwise, the input signals to the other address control circuits215(v) whose latches are not already set [that is, circuits 215(v) where(v) is an even number but not divisible by four] are negated.

At this point, the latches again latch the input signals. This processis repeated for the third and fourth RTR ADRS router address bits whichare received. It will be appreciated that in response to each RTR ADRSrouter address bit, the latches in one-half of the address controlcircuits 215(v) are set. Accordingly, after four RTR ADRS router addressbits are received and processed by the circuit depicted in FIG. 9, thelatch of only one address control circuit will be cleared, and theothers will all be set. In the example above in which the RTR ADRSrouter address bits have the encoding "0000", only the latch of addresscontrol circuit 215(0) will be cleared, and that address control circuitwill assert the ADRS(0)EN(0) signal. The operations in response to otherencodings of the RTR ADRS router address bits are similar, except thateach different encodings enables latches of a different address controlcircuit 215(v) to remain cleared, enabling that address control circuitto assert its ADRS(v)EN(0) signal.

C. Description of Specific Circuits

With this background, specific circuits for various portions of theswitching chip depicted in FIG. 9 will be described. FIG. 10A depicts acircuit of an input (u) circuit, FIGS. 10B-1 and 10B-2, togethercomprising FIG. 10B, depict a circuit of an address control circuit215(v) and its related coupling group 216(v). FIG. 10C depicts a circuitrelating to a data line 220(v)(w) that comprises part of an outputterminal group 217(v).

In the following description, the index (u), which is used in thedescription of FIG. 9, has not been used in connection with the signalnames and reference numerals.

i. Input Circuit 211

With reference to FIG. 10A, an input circuit 211 receives an IN inputsignal on an input line 230. The IN input signal is coupled to twolatches, namely a miss latch 231 and a break latch 232, to an inverter233 and to one input terminal of an AND gate 234. In response to the INinput signal, the inverter 233 generates an ADRS signal which is coupledto the address control circuit depicted in FIG. 10B-1.

As has been described above in connection with FIG. 8B, the first signalbit received by the switching chip, and thus the first input signalreceived by the input circuit 211, is a protocol bit, labeled P in FIG.8B. If the bit has a binary value of one, that is, if the IN signal isasserted during the P bit time, a message is being received through theinput circuit 211. If, on the other hand, the bit has a binary value ofzero, that is, if the IN signal is negated during the P bit time, nomessage is being transferred through input the input circuit. Miss latch231 latches the condition of the IN signal during the P bit time, whichis defined by an LD MISS LTH load miss latch timing signal which derivedfrom a RTR CTRL router control signal (not shown) from the array controlunit 10 (FIG. 1).

In particular, the RTR CTRL router control signal is transmitted to theswitching chips of the first stage 201 (FIG. 8A) of the routing network30 (FIG. 2). When the switching chips 201(0) through 201(63) of thatstage have retired all of the bits of the first router control field 205(FIG. 8B) and are prepared to transmit to the next stage 202, theytransmit a RTR CTRL NXT router control next stage signal to theswitching chips 202(0) through 202(63). This process is repeated asbetween switching chips of stage 202 and crossbar stage 203, and asbetween chips of crossbar stage 203 and the input terminals of PE chips20 (FIG. 2). Thus, the array control unit 10 may initiate a transferthrough the routing network with the single RTR CTRL router controlsignal coupled only to the first switching stage, with each stagecontrolling timing of the next stage. This arrangement simplifiesexpansion of the routing network, as the array control unit does nothave to be informed of any minimum or maximum number of switchingstages.

The miss latch 231 includes two inverters 233 and 234, with the outputof inverter 233, which, provides a MISS signal, being connected to theinput of inverter 234. The output of inverter 234 is connected through apass transistor 235 to the input of inverter 233. The IN input signal isalso coupled to the input of inverter 233 through a pass transistor. Thepass transistors are, in turn, controlled by the LD MISS LTH load misslatch timing signal. Transistor 236 is on, that is, conducting, when theLD MISS LTH load miss latch timing signal is asserted, and otherwise isoff. Transistor 235, on the other hand, is controlled by the complementof the LD MISS LTH load miss latch timing signal, and is on, that is,conducting, when the LD MISS LTH load miss latch timing signal isnegated, and otherwise off.

Thus, when the LD MISS LTH load miss latch timing signal is asserted,transistor 236 is turned on and transistor 235 is turned off. Transistor236 couples the IN input signal to the input of inverter 233. The MISSsignal from inverter 233 is the complement of the received IN signalwhile the LD MISS LTH load miss latch timing signal is asserted.Inverter 234 receives the MISS signal from inverter 233 and complementsit again. When the LD MISS LTH load miss latch timing signal is againnegated, at the end of the time defining the P protocol bit (FIG. 8B),transistor 236 turns off, isolating the input of inverter 233 from theIN input signal.

At the same time, however, transistor 235 turns on, coupling the outputsignal from inverter 234 to the input of inverter 233. Since the outputsignal from inverter 234 has the same condition as the IN signal whilethe LD MISS LTH load miss latch timing signal was asserted, the MISSsignal from inverter 233 remains in the same condition; that is, the INsignal is latched in the miss latch 231. The MISS signal represents thecomplement of the P bit, that is, the complement of the IN input signalwhen the LD MISS LTH load miss latch timing signal was last asserted.When the LD MISS LTH load miss latch timing signal is next asserted,transistor 235 is turned off, isolating the input of inverter 233 fromthe output of inverter 234 and permitting the IN input signal to becoupled to the input of inverter 233 through transistor 236.

If the MISS signal is asserted, the IN signal during the P protocol bittime was negated, and so no message is being received through the inputcircuit 211. If, on the other hand, the MISS signal is negated, the INsignal was asserted during the P protocol bit time, and a message isbeing received through the input circuit 211.

The break latch 232 is constructed in a manner similar to miss latch 232and operates in the same way in response to a LD BRK LTH IN load breaklatch in timing signal from the array control unit 10 (FIG. 1). Thebreak latch is used to latch the IN input signal while the communicationpath from the preceeding chip is tested to determine if it has beendisrupted due to a broken wire or otherwise, as described below. If thecommunication path is disrupted, the break latch 232 asserts a BREAKsignal and negates its complement, a NO BRK no break signal. The BREAKsignal is coupled to an input of OR gate 240 which generates a BRK OROUT break OR out signal. Or gate 240 receives a BRK OR IN break OR inputsignal from a similar circuit in an input circuit 211 on the switchingchip or, as described below in connection with FIG. 10C, an outputterminal group circuit 217(v). The OR gates 240 are connected in an ORchain which drives an illumination device (not shown) so that, if anycommunications path to which the chip is connected is disrupted, theillumination device is energized.

The NO BRK no break signal is coupled to a NAND gate 241, to which thecomplement of the MISS signal is also coupled. If either a misscondition occurs, or the communication pathway to input terminal 230 isdisrupted, NAND gate 240 asserts a BLOCK signal which is coupled to theaddress control circuits 216(v), as described below in connection withFIG. 10B-1. In brief, the BLOCK signal, when asserted, inhibits theaddress control circuits connected to the input circuit 211 fromasserting the ADRS(v)EN(u) coupling circuit enabling signals.

As described below in connection with FIG. 10B-2, a CHP MISS chip misssignal is asserted if no coupling circuit in a switching circuit 210(u)is enabled to couple the DAT(u) data signal onto a data output line 220.An OR gate 242 is energized to assert a DAT BLK data blocking signalwhen either the BLOCK signal or the CHP MISS chip miss signal isasserted. The complement of the DAT BLK data blocking signal isgenerated by an inverter 243 whose output is coupled to AND gate 234.Thus, if the DAT BLK data blocking signal is asserted, the AND gate 234is disabled, inhibiting the AND gate 234 from coupling the IN inputsignal received on line 230 to line 214 as the DAT signal.

AND gate 234 is also disabled by an inverter 245 when an ACK TIME signalis asserted, which occurs when the ACK acknowledge bit of a message (seeFIG. 8B) is expected to be transferred from the destination PE to theoriginating PE. The ACK TIME signal is provided by the array controlunit 10. Thus, when the ACK TIME signal is asserted, the AND gate 234isolates line 214 from the signal on line 230. Conversely, the AND gateis enabled to couple the IN signal on line 230 onto line 214 as the DATsignal when both the ACK TIME signal is negated, which occurs at timesother than during the acknowledgement bit time, and when the DAT BLKdata block signal is not asserted, indicating that a coupling module isenabled to couple the DAT signal onto an output data line 220(v)(w).

The remaining circuitry on FIG. 10A performs two functions. Inparticular, during the time for transferring the message acknowledgementbit, which occurs when the ACK TIME acknowledgement time signal isasserted, the signal representing the ACK bit is received by the inputcircuit over line 214. Circuitry is provided to couple the signal ontoline 230 during the acknowledgement time. In addition, circuitry isprovided to perform the communications path continuity test, whichoccurs in response to a PR RTR IN(L) precharge router in (asserted low)signal and a DR LOW RTR IN(L) drive low router in signal, both of whichare provided by the array control unit 10 (FIG. 1).

During the acknowledgement bit transfer time, the ACK TIME signal isasserted, which disables AND gate 234. The PR RTR IN(L) precharge routerin (asserted low) and the DR LOW RTR IN(L) drive low router in (assertedlow) signals are both negated, and thus are at a high voltage level,thus enabling one input each of NAND gates 246 and 247 and two inputs aNAND gate 250. Since the ACK TIME signal is asserted, the inverter 245couples a low input signal to the third input of NAND gate 250. Theoutput of NAND gate 250 thus is at a high voltage level, which enablesan AND gate 251, whose output is connected to line 230.

The DAT signal on line 214, which at this time represents the ACKacknowledgement bit, is complemented in an inverter 252, inverted twicethrough the two enabled NAND gates 246 and 247, and complemented againin a second inverter 253, whose output is coupled to the second input ofAND gate 251. Thus, the signal provided by inverter 253 to AND gate 251has the same sense as the DAT signal on line 214. Since AND gate 251 isenabled by the NAND gate 250, it couples the signal from inverter 253onto line 230. Thus, during the acknowledgement bit time, the DATsignal, which represents the ACK acknowledgement bit, is coupled fromline 214 to line 230, that is, through the input circuit 211.

At other times than the acknowledgement bit time, however, the ACK TIMEsignal is negated. Inverter 245 thus couples a high input signal to NANDgate 250. Since the other input signals to NAND gate 250 are normallynegated and thus at a high voltage level, the output of NAND gate 250 islow, thus disabling AND gate 251. Accordingly, as long as the PR RTRIN(L) precharge router in (asserted low) and DR LOW RTR IN(L) drive lowrouter in (asserted low) signals are negated, during times other thanthe acknowledgement bit time, AND gate 251 is inhibited from couplingthe signal from inverter 253 to line 230.

The communications path continuity test is performed by circuitry inboth the input circuit 211 depicted in FIG. 10A and the output terminalgroup described below in connection with FIG. 10C, and will be describedbelow in connection with FIG. 10C.

ii. Address Control Circuit 215 And Coupling Circuit 216

FIGS. 10B-1 and 10B-2 depict the circuit diagram of an address controlcircuit and four coupling modules 216(0) through 216(3) [generallyidentified by reference numeral 216(w)] comprising one coupling circuit216.

a. Address Control Circuit 215

As described above, the address control circuit 215 includes a latch 260comprising two inverters 261 and 262. The output of inverter 261 isconnected to the input of inverter 262, and the output of inverter 262is connected to a pass transistor 263, which, when it is turned on by anADRS HOLD address hold timing signal, couples the output signal frominverter 262 to the input of latch 261. Another pass transistor 264 alsocontrolled by the ADRS HOLD address hold timing signal, couples theoutput signal from inverter 262 to control two pass transistors 265 and266, and to an inverter 267 which controls a pass transistor 270.

An EN RTR enable router timing signal from the array control unit 10(FIG. 1), when negated, through an inverter 268 turns on a transistor269. This places a high signal at the input of inverter 261. When therouter chip is enabled, the array control unit 10 asserts the EN RTRenable router timing signal, turning off transistor 269 and isolatingthe input of inverter 261 from the high input. Thus, immediately afterthe EN RTR enable router signal is asserted, latch 260 is in thecondition such that the input of inverter 261, and thus the output ofinverter 262 are high. In addition, the ADRS HOLD address hold timingsignal is asserted. In this condition, the high output signal frominverter 262 enables the pass transistors 265 and 266 to be in the on,or conducting, condition, and pass transistor 270 to be in the off, ornon conducting condition by inverter 267.

In response to the assertion of an ADRS TIME address time signal, whichis asserted to define each RTR ADRS router address bit (see FIG. 8B) inthe IN input signal (FIG. 10A), a pass transistor 271 is turned on,allowing the ADRS address signal from inverter 233 (FIG. 10A) to becoupled through transistor 271 and transistor 265. While the ADRS TIMEsignal is asserted, the ADRS HOLD address hold signal is negated,turning off transistors 263 and 264. While transistor 264 is turned off,transistors 265 and 266 and inverter 267 are held in the condition theywere in by residual charge which remains on their inputs. However, theADRS signal overcomes any residual charge on the input to inverter 261,and thus inverter 261 couples, at its output, a signal corresponding tothe complement of the ADRS address signal.

Since at this point pass transistor 266 is maintained on by its residualcharge, it couples the output signal from inverter 261 to a line 272,which carries an ADRS NXT address next signal to the next addresscontrol circuit 215 in the switching circuit 210 (see FIG. 9). It willbe appreciated that the inverter 261 included in the latch circuit 260,which comprises the latch described above in connection with FIG. 9, isalso used as the inverter described above in connection with FIG. 9.

It will also be appreciated that, if the ADRS address signal is in ahigh condition while the ADRS TIME address time signal is asserted, theoutput signals from inverters 261 will be maintained in the conditionswhich they had when the address control circuit was initialized inresponse to the EN RTR enable router initialization signal. However, ifthe ADRS address signal is in a low condition, the input to inverter 261will be low, as will the output of inverter 262. Thus, when the ADRSTIME signal is next negated, and the ADRS HOLD address hold signal isasserted, the output of inverter 262 will be in a low condition.Transistor 264, which is turned on by the ADRS HOLD address hold signal,couples the low output signal to transistors 265 and 266, turning themoff, and to inverter 267, turning on transistor 270.

Since transistor 270 is turned on, the ADRS address signal is coupledonto line 272 as the ADRS NXT signal to the next address controlcircuit, bypassing the inverter 261 as described above in connectionwith FIG. 9. In addition, since transistor 265 is turned off, the ADRSaddress signal is blocked in the path to inverter 261 when the ADRS TIMEaddress timing signal is next asserted, thereby isolating the latch 260from the ADRS address signal. It will be appreciated that, once a lowADRS signal is received by latch 260, the latch 260 remains isolatedfrom the ADRS address signal until the EN RTR enable routerinitialization signal is asserted, which enables transistor 265 to beturned on.

The output of transistor 261 is coupled to one input of a NOR gate 273.If the latch 260 receives four consecutive asserted signals (that is,signals at a high voltage level) through transistors 265 every time theADRS TIME signal is asserted, which corresponds to the four RTR ADRSbits that are retired by the switching chip on which the circuit isresident (see FIG. 8B), the output signal from inverter 261 is in a lowvoltage condition. If the BLOCK signal from input circuit 211 (FIG. 10A)is negated (in a low voltage condition), NOR gate 273 is energized toassert (high) an ADRS EN enabling signal, which corresponds to theADRS(v)EN(u) enabling signal described above in connection with FIG. 9.

Thus, if the address control circuit 215 receives an asserted ADRSaddress signal during the four consecutive asserted ADRS TIME addresstime enabling signals which define the four RTR ADRS bits, and if the Pprotocol bit (see FIG. 8B) has been received, which ensures that theBLOCK signal will be negated, the ADRS EN address enabling signal willbe asserted. Otherwise, the ADRS EN address enabling signal will benegated.

b. Coupling circuit 216

The coupling circuit 216 includes four coupling modules 216(0) through216(3) which are depicted in FIGS. 10B-1 and 10B-2. Coupling module216(0) is shown in FIG. 10B-1 and the other coupling modules aredepicted on FIG. 10B-2. Since the four coupling modules are similar,only coupling module 216(0) will be described in detail.

With reference to FIG. 10B-1, coupling module 261(0) receives the ADRSEN address enabling signal from the address control circuit 215 and aGRP EN 0 IN (L) group enable (asserted low) signal [which corresponds tothe GRP(v)EN(w) group enable signal described above in connection withFIG. 9]. If the ADRS EN address enable signal is asserted (high) but theGRP EN 0 IN (L) group enable signal is assserted (that is, also low), anAND gate 280 is asserted , which turns on a transistor 281, which inturn couples the DAT signal from FIG. 10A onto output data line220(v)(0).

The asserted (low) GRP EN 0 IN (L) group enable in signal iscomplemented by an inverter 282 to enable one input of a NAND gate 283.The second input of NAND gate 283 is the complement of the ADRS ENaddress enable signal from an inverter 284. If the ADRS EN signal isasserted, the output signal from inverter 284 is negated. As will bedescribed below, with the input signals to NAND gate 283 in thatcondition, the NAND gate 283 generates a high (negated) GRP EN 0 OUT (L)group enable out (asserted low) output signal, which inhibits thesucceeding coupling modules connected to the data output line 220(v)(0)from coupling data signals onto the data line.

When the ADRS EN address enable signal from address control circuit 215is negated, the GRP EN OUT (L) group enable out signal has the samecondition as the GRP EN 0 IN (L) group enable in signal. If the ADRS ENaddress enabling signal is negated, the AND gate 280 is disabled andtransistor 281 is maintained in the off condition. Thus, the transistor281 blocks the DAT signal from being coupled onto dataline 220(v)(0). Atthe same time, the negated ADRS EN signal is complemented to a highvoltage condition by inverter 284, which enables one input of NAND gate283. If the GRP EN 0 IN (L) group enable in signal is asserted (low),inverter 282 couples a high signal to the other input of NAND gate 283.With the input signals in that condition, NAND gate 283 asserts the GRPEN 0 OUT (L) group enable out signal, allowing another coupling module216(0) in another switching circuit 210 (FIG. 9) to use the data line220(v)(0).

If, on the other hand, the GRP EN 0 IN (L) group enable in signal isnegated (high), the inverter 282 couples a low input signal to the NANDgate 283, enabling the NAND gate to transmit a high, or negated, GRP EN0 OUT (L) group enable signal. Thus, when the ADRS EN address enablesignal is negated, the asserted or negated condition of the GRP EN D OUT(L) group enable out signal to the succeding coupling module connectedto the data line 220(v)(0) is the same as the condition of the receivedGRP EN 0 IN (L) group enable in signal.

If the ADRS EN address enable signal is asserted, but the GRP EN 0 IN(L) group enable signal is negated (high), an AND gate 285 is energizedto assert an EN 1 enabling signal, which is coupled to coupling module216(1) and is used by that coupling module in the same way that thecoupling module 216(0) uses the ADRS EN address enable signal. If theGRP EN 0 IN (L) group enable signal is asserted (low), the AND gate 285is disabled and the EN 1 enabling signal is negated. Thus, it will beappreciated that if a coupling module 216(w) is enabled by the ADRS ENaddress enable signal from the address control circuit 215, or by the ENW ("W" being 1 or 2), if it is inhibited from coupling the DAT datasignal onto its respective data line 220(v)(w) by the negated GRP EN WIN (L) group enable signal, it will assert the EN W (W being 1, 2, or 3)to enable the next coupling module to its right. However, if a couplingmodule 216(w) is enabled by both the ADRS EN address enable or EN Wenabling signals and the GRP EN W IN (L) group enable signal, it willnegate the EN W signal to the next coupling module to its right.

Also depicted in FIGS. 10B-1 and 10B-2 is circuitry for terminating thedata lines 220(v)(0) through 220(v)(3) to ensure that the voltage levelof the signals on those lines does not float if no coupling modulecouples its associated DAT data signals onto the data lines. If the GRPEN W OUT (L) group enable out signal from the last coupling module216(0) associated with a data line 220(v)(0) is asserted, that is, in alow voltage condition, an inverter 290 energizes a transistor 291, whichcauses a ground level signal to be placed on the data line 220(v)(0).This ground level signal is coupled to the next switching chip in therouting network, or to the routing network input terminal of a PE chip.The ground level signal provides a negated signal during the P protocolbit time (FIG. 8B), enabling the miss latch 231 to be set in the inputcircuit 211.

As has been mentioned, the switching chips described in connection withFIGS. 9 and 10A through 10C can also perform crossbar switching, andthus are useful in the crossbar switching stage 203 (FIG. 8A). This isaccomplished if only the GRP EN 0 IN (L) group enabling signal isasserted by the output terminal group and the other signals, namely theGRP EN 1 IN (L) through GRP EN 3 IN (L) group enabling signals arenegated.

c. Output Terminal Group

Each output data line 220(v)(w) has an associated output circuit217(v)(w) in the output terminal group 217(w) (FIG. 9), which alsogenerates the GRP EN W (L) group enable signal. The circuit in an outputterminal group associated with one output data line will be described inconnection with FIG. 10C.

With reference to FIG. 10C, the output data line is connected to aninverter 300, which complements the DATA OUT signal and couples it toone input of a NAND gate 301. The NAND gate is controlled by a PR RTROUT (L) precharge router out (asserted low) signal which is used inconnection with the communication path continuity test described below.The PR RTR OUT (L) precharge router out signal is normally negated, thatis, at a high voltage level, and in that condition, the output signalfrom the NAND gate 301 is the same as the DATA OUT signal.

The output of NAND gate 301 is coupled to a second NAND gate 302, whichis controlled by a DR LOW RTR OUT (L) drive low router out (assertedlow) signal, which also is used in connection with the communicationpath continuity test. The DR LOW RTR OUT (L) drive low router out(asserted low) signal is also normally negated, thus providing an outputsignal which is the complement of the output of NAND gate 301. Theoutput of NAND gate 302 is again complemented in an inverter 303 whichcouples a DATA NXT CHP data next chip output signal onto an output line304, which is connected to the next switching stage or to the routerinput of a PE chip pair. It will be appreciated that, as long as the PRRTR OUT (L) precharge router out and DR LOW RTR OUT (L) drive low routerout signals are negated, the DATA NXT CHP data next chip output signalprovided by inverter 303 corresponds to the DATA OUT signal input toinverter 300.

The output line 304 is also connected to a break latch 305, which isconstructed in the same way as break latch 232 (FIG. 10A) and operatesin response to a signal from line 304 and a LD BRK LTH OUT load breaklatch out timing signal from array control unit 10 (FIG. 1). Break latch305 is used in connection with the communications path continuity testdescribed below. If there is no disruption of the communications path,break latch 305 asserts a NO BRK OUT (L) no break out (asserted low)signal, which enables one input of an OR gate 306. OR gate 306 is alsoenabled by a DRV DIS output driver disable signal from other circuitry(not shown) in the array processor which controls the communicationspaths through routing network 30 (FIG. 2). If the DRV DIS driver disablesignal is negated (low) and the NO BRK OUT (L) no break out signal isasserted (low), OR gate 306 generates the GRP EN W (L) group enablesignal, which is coupled to a coupling module as depicted on FIGS. 10B-1and 10B-2. The NO BRK OUT (L) no break out signal is also coupled to anOR gate 307, which performs the same function as OR gate 240 describedin connection with the input circuit 211 (FIG. 10A).

During times other than the ACK acknowledgement bit time (FIG. 8A), theACK TIME signal is negated. When the ACK TIME signal is negated, an ANDgate 310 is disabled, isolating the data line 220(v)(w) from the outputline 304. However, when the ACK TIME signal is asserted, AND gate 310 isenabled to couple the signal received from downstream circuitry, eithera switching chip in a succeeding switching stage (FIG. 8A) or a PE chip,to the data line 220(v)(w). With reference to FIG. 10B-1, it will beappreciated that the transistor 281, when enabled by energized AND gate280 to couple the DAT data signal from line 214 (FIG. 10A) onto dataline 220(v)(0), is also capable of coupling the signal representing theACK acknowledgement bit in the reverse direction from line 220(v)(0)onto line 214.

If the GRP EN N (L) group enable signal is negated (that is, in a highvoltage condition), a transistor 313 is turned on. While the ACK TIMEtiming signal is negated, an inverter 312 also turns on anothertransistor 311. Transistor 311 and 313 are connected in series betweenline 220(v)(w) and a ground levelsignal represented by the legend Vss.Thus, when the two transistors are turned on, a ground level signal inplaced on data line 220(v)(w) so that the voltage level of the signal onthe data line does not float while the GRP EN N (L) group enable signalinhibits the coupling modules 216(v)(w) from coupling DAT data signalsonto the data line. When the ACK TIME acknowledgment timing signal isasserted, inverter 312 turns off the transistor 311, which isolates thedata line 220(v)(w) from the ground level signal.

d. Communications Path Continuity Test

As has been mentioned the input circuit 211 depicted in FIG. 10A and theoutput circuit 217(v)(w) depicted in FIG. 10C includes circuitry forperforming a communications path continuity test between switching chipsand for inhibiting transfers in the event of a disruption in thecontinuity of the communications path. With reference to FIGS. 10A and10C, the test is performed in three stages, each initiated by means of atiming signal. First, the communications line, that is, line 230 or line304 (it will be appreciated that line 304 corresponds to line 230 in theswitching chip of the next stage), is precharged to a high voltage levelby either the input circuit 211 or the output circuit. This occurs inresponse to the PR RTR IN (L) precharge router in (low) signal, if theprecharge operation is performed by the input circuit 211, or the PR RTROUT (L) precharge router out (low) signal if the precharge operation isperformed by the output circuit.

After the communications line has been precharged, the other circuit(that is, the output circuit if the input circuit 211 precharged theline, or the input circuit 211 if the output circuit precharged theline) places a signal on the communications line which drives the lineto a low voltage condition. This occurs in response to the DR LOW RTR IN(L) drive low router in (low) or DR LOW RTR OUT (L) drive low router out(low) signal. Finally, the break latch 232 or 305 in the circuit whichinitially precharged the line is enabled by a LD BRK LTH IN or LD BRKLTH OUT signal latches the state of the signal on the communicationsline. If the low signal reaches the break latch the communications lineis continuous and the break latch asserts a NO BRK IN or NO BRK OUT nobreak signal. On the other hand, if the low signal fails to reach thebreak latch, the communications path is disrupted, and so the NO BRK INor NO BRK OUT signal will be negated. The test performed by both theinput circuit 211 and the output circuit at both ends of thecommunications path so that break latches in both circuits can beproperly conditioned.

In the following description, the output line 304 is taken as the otherend of input line 230; otherwise stated, input circuit 211 is in thenext switching stage from the output circuit depicted in FIG. 10C. Withthis background, with reference to FIGS. 10A and 10C, when the PR RTR IN(L) precharge router in (low) signal is asserted (low), the outputsignal from NAND gate 246 is driven high. Since the DR LOW RTR IN (L)drive low router in (low) signal is negated (that is, high) NAND gate246 couples a low signal to inverter 253, which inverted to a highsignal and coupled to one input of AND gate 251.

Since the PR RTR IN (L) precharge router in signal is asserted (low),NAND gate 250 also couples a high signal to the second input of AND gate251. As a result, AND gate 251 is energized, forcing line 230 to a highvoltage level.

The output circuit depicted in FIG. 10C then places a low signal on line304. This occurs when the DR LOW RTR OUT (L) drive low router out (low)signal is asserted. When that occurs, NAND gate 302 transmits a highsignal, which is complemented to a low voltage level by inverter 303.

Returning to FIG. 10A, when the signal from the output circuit depictedin FIG. 10C should have reached the input circuit, the LD BRK LTH INsignal is asserted, enabling break latch 232. If the communications linebetween the input circuit 211 and the output circuit is not disrupted,the signal on line 230 is at a low voltage level, which is inverted bythe break latch circuit to provide the asserted (high) NO BRK IN nobreak in signal. On the other hand, if the communications path isdisrupted, the low signal from inverter 303 will not reach the breaklatch. Instead, the signal on line 230 will be high, which is invertedto form the negated (low) NO BRK IN no break in signal.

Similar operations are performed to load the break latch 305 in theoutput circuit depicted in FIG. 10C. In particular, when the PR RTR OUT(L) precharge router out (low) signal is asserted (low), the outputsignal from NAND gate 301 is driven high. Since the DR LOW RTR OUT (L)drive low router out (low) signal is negated (that is, high) NAND gate302 couples a low signal to inverter 303, which is inverted to a highsignal to precharge line 304.

The input circuit depicted in FIG. 10A then places a low signal on line230. This occurs when the DR LOW RTR IN (L) drive low router in (low)signal is asserted. When that occurs, NAND gate 247 transmits a highsignal, which is complemented to a low voltage level by inverter 253.The low signal from inverter 253 disables AND gate 251, causing a lowsignal to be placed on line 230.

Returning to FIG. 10C, when the signal from the input circuit depictedin FIG. 10A should have reached the output circuit, the LD BRK LTH OUTsignal is asserted, enabling break latch 305. If the communications linebetween the input circuit 211 and the output circuit is not disrupted,the signal on line 304 is at a low voltage level, which is inverted bythe break latch circuit 305 to provide the asserted (high) NO BRK OUT nobreak out signal. On the other hand, if the communications path isdisrupted, the low signal from AND gate 251 will not reach the breaklatch 305. Instead, the signal on line 304 will be high, which iscoupled through latch 305 to provide the negated NO BRK OUT(L) no breakout (asserted low) signal.

The foregoing description has been limited to a specific embodiment ofthis invention. It will be apparent, however, that variations andmodifications may be made to the invention, with the attainment of someor all of the advantages of the invention. Therefore, it is the objectof the appended claims to cover all such variations and modifications ascome within the true spirit and scope of the invention.

What is claimed as new and desired to be secured by Letters Patent ofthe United States is:
 1. A routing network for transferring messages,each message including an address and data, among a plurality ofprocessing elements in an array processing system, said routing networkcomprising: I. a series of initial switching stages each including:A. aplurality of switching elements, each comprising a plurality of outputterminal groups, each group comprising a plurality of output terminalmeans, each output terminal group being identified by a decoding of aselected portion of an address; B. a plurality of data lines eachconnected to an output terminal means, wherein all of said data lines,from each said switching element in said switching stage, connected tosaid output terminal means belonging to said output terminal groupidentified by the same said address decoding, are also connected to thesame switching element in the next switching stage; C. circuit means forreceiving a message and coupling it onto a selected one of said datalines, said circuit means including:i. input circuit means for receivingan input message and having an address output terminal for transmittingsaid address and a data output terminal for transmitting said data; ii.a plurality of address control circuits for decoding the address fromthe address output terminal of said input circuit means, each addresscontrol circuit generating an address enabling signal in response to apredetermined encoding of the address; iii. a plurality of couplingmeans groups each connected to an address control circuit for receivingan enabling signal therefrom, each coupling means group including acoupling means each connected to a data line, one of said coupling meanswithin a coupling means group coupling said data from said input circuitmeans onto a data line in response to the receipt of said addressenabling signal from said address circuit means; and II. a crossbarswitch stages for receiving messages from the output terminal groups ofthe last of the series of initial switching stages, for performing aswitching operation in connection with the messages, and for couplingthe switched messages to the processing elements.
 2. A routing networkas defined in claim 1 in which the coupling means within said couplingmeans group are interconnected in a predetermined order, with a highorder coupling means being connected to receive said address enablingsignal from said address control circuit, and each coupling means beingconnected to transmit the address enabling signal to the next lowerorder coupling means if it does not couple said data from said inputcircuit means onto a data line.
 3. A routing network as defined in claim2 in which each output terminal means includes means for generating acoupling enabling signal in response to a selected condition, eachcoupling means including:A. coupling enabling signal receiving means forreceiving a coupling enabling signal; B. data coupling means responsiveto the receipt of a coupling enabling signal and an address enablingsignal for coupling data from said input circuit means onto a data line;and C. coupling enabling signal transmitting means connected to saidcoupling enabling signal receiving means and said data coupling meansfor transmitting the coupling enabling signal to another coupling meansin response to the receipt of a coupling enabling signal if said datacoupling means does not couple data from said input circuit means onto adata line.
 4. A routing network as defined in claim 3 in which said datacoupling means comprises:A. pass transistor means connected to saidinput circuit means and said data line for transferring data betweensaid input circuit means, data output terminal and said data line inresponse to a pass transistor control signal; B. control signalgenerating means connected to receive said coupling enabling signal andsaid address enabling signal for generating said pass transistor controlsignal in response to the coincidence of said address enabling signaland said coupling enabling signal.
 5. A routing network as defined inclaim 4 in which said coupling means further includes coincidence meansconnected to receive said coupling enabling signal and said addressenabling signal for transmitting an address enabling signal to said nextlower order coupling means in response to the coincidence of theassertion of said address enabling signal and the negation of saidcoupling enabling signal.
 6. A routing network as defined in claim 1 inwhich said input circuit means further includes block signal generatingmeans for generating a block signal in response to a selected condition,said address control circuits being disabled in response to the receiptof a block signal.
 7. A routing network as defined in claim 6 in whicheach said address control circuit includes:A. block signal receivingmeans connected to said input circuit means for receiving said blocksignal; B. address decoding means connected to said input circuit meansfor generating an address decode signal in response to a selecteddecoding of the address from said input circuit means; C. addressenabling signal generating means connected to said block signalreceiving means and said address decoding means for generating saidaddress enabling signal in response to the receipt of said addressdecode signal and the absence of said block signal.
 8. A routing networkas defined in claim 6 in which said address includes a protocol bit,said input circuit including protocol means for detecting said protocolbit and for generating a mess signal if said protocol bit is notdetected, said block signal generating means being connected to saidprotocol means for generating said block signal in response to thegeneration of said miss signal.
 9. A routing network as defined in claim6 in which said each input circuit means is connected to a messagetransfer wire for receiving a message thereover, said input circuitincluding broken wire detection means for detecting the condition of themessage transfer wire and generating a broken wire signal in responsethereto, said block signal generating means being connected to saidbroken wire detection means for generating said block signal in responseto the generation of said broken wire signal.
 10. A routing network asdefined in claim 9 in which each said output terminal means is alsoconnected to a message transfer wire, said message transfer wiretransferring messages between said output terminal means of one of saidinitial switching stages and said input means of succeeding one of saidinitial switching stages, said output terminal means also includingbroken wire detection means,A. each said broken wire detection meansincluding:i. broken wire signal generating means connected to saidmessage transfer wire for transmitting a signal over said messagetransfer wire; ii. broken wire latch means connected to said messagetransfer wire for latching a signal received over said message transferwire and generate said broken wire signal in response to the conditionof the latched signal; and B. said routing network further includingbroken wire control means connected to said broken wire signalgenerating means and said broken wire latch means in each broken wiredetection means for separately enabling said broken wire generatingmeans in said input means and said output terminal means to transmit asignal on the message transfer line, and the broken wire latch means inthe other of input means and said output terminal means to latch thecondition of the signal on the message transfer line.
 11. A routingnetwork as defined in claim 1 in which said address comprises apredetermined number of serial address bit signals, and wherein saidaddress control circuits in each circuit means are connected in seriesordered relationship from the circuit means' input circuit means, eachaddress control circuit including:A. address bit signal receiving meansfor receiving an address bit signal from said input circuit or fromanother address control circuit in said series order;B. address bitcomplement means connected to said address bit signal receiving meansfor generating a complement address bit signal in response to theaddress bit signal received by said address bit signal receiving means;C. multiplexer means connected to said address bit signal receivingmeans and said address bit complement means for selectively couplingeither the address bit signal or the complement address bit signal tothe next address control circuit in the series order; and D. controlmeans connected to said address bit signal receiving means and saidmultiplexer means for controlling said multiplexer means, said controlmeans including:i. address bit signal coupling means connected to saidaddress bit signal receiving means for coupling said address bit signalreceived by said address bit signal receiving means; ii. latch meansconnected to said address bit signal coupling means and said multiplexermeans for latching the address bit signal received from said address bitsignal coupling means and for controlling said address bit signalcoupling means and said multiplexer means in response thereto, saidlatch means controlling said address bit signal coupling means to coupleaddress bit signals thereto while said address bit signals have aselected condition and contemporaneously enabling said multiplexer meansto couple the complement address bit signal to the next address controlcircuit, and to disable said address bit signal coupling means onreceipt of an address bit signal having another condition andcontemporaneously enabling said multiplexer means to couples saidaddress bit signal to the next address control circuit.
 12. A routingnetwork as defined in claim 1 for transferring messages each includingan acknowledge bit in response to an acknowledge transfer enablingsignal, each output terminal means includes means connected to its dataline and an output terminal for coupling data signals from said dataline to said output terminal, and output gate means connected to saidoutput terminal and said data line for coupling said acknowledge bitfrom said output terminal to said data line in response to saidacknowledge transfer enabling signal, each of said input circuit meansincluding input gate means connected to said data output terminal andfor coupling said acknowledge bit from said data output terminal to saidinput terminal in response to the receipt of said acknowledge transferenabling signal.
 13. A routing stage for transferring messages among aplurality of processing elements in a routing network, the routing stageincluding:A. a plurality of switching elements, each comprising aplurality of output terminal groups each comprising a plurality ofoutput terminal means, each output terminal group being identified by adecoding of a selected portion of an address; B. a plurality of datalines each connected to an output terminal means, wherein all said datalines, from each said switching element in said switching stage,connected to said output terminal means belonging to said outputterminal group identified by the same said address decoding, are alsoconnected to same said switching element in the next subsequent routingstage; C. circuit means for receiving a message and coupling it onto aselected one of said data lines, said circuit means including:i. inputcircuit means for receiving an input message and having an addressoutput terminal for transmitting said address and a data output terminalfor transmitting said data; ii. a plurality of address control circuitfor decoding the address from address output terminal of said inputcircuit means each address control circuit generating an addressenabling signal in response to a predetermined encoding of the address;and iii. a plurality of coupling means groups each connected to anaddress control circuit for receiving an enabling signal therefrom, eachcoupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data from said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means.
 14. A routing network as defined in claim 13in which the coupling means within said coupling means group areinterconnected in a predetermined order, with a high order couplingmeans being connected to receive said address enabling signal from saidaddress control circuit, and each coupling means being connected totransmit the address enabling signal to the next lower order couplingmeans if it does not couple said data from said input circuit means ontoa data line.
 15. A routing stage as defined in claim 14 in which eachoutput terminal means includes means for generating a coupling enablingsignal in response to a selected condition, each coupling meansincluding:A. coupling enabling signal receiving means for receiving acoupling enabling signal; B. data coupling means responsive to thereceipt of a coupling enabling signal and an address enabling signal forcoupling data from said input circuit means onto a data line; and C.coupling enabling signal transmitting means connected to said couplingenabling signal receiving means and said data coupling means fortransmitting the coupling enabling signal to another coupling means inresponse to the receipt of a coupling enabling signal if said datacoupling means does not couple data from said input circuit means onto adata line.
 16. A routing network as defined in claim 15 in which saiddata coupling means comprises:A. pass transistor means connected to saidinput circuit means and said data line for transferring data betweensaid input circuit means' data output terminal and said data line inresponse to a pass transistor control signal; B. control signalgenerating means connected to receive said coupling enabling signal andsaid address enabling signal for generating said pass transistor controlsignal in response to the coincidence of said address enabling signaland said coupling enabling signal.
 17. A routing network as defined inclaim 16 in which said coupling means further includes coincidence meansconnected to receive said coupling enabling signal and said addressenabling signal for transmitting an address enabling signal to said nextlower order coupling means in response to the coincidence of anassertion of said address enabling signal and the negation of saidcoupling enabling signal.
 18. A routing stage as defined in claim 13 inwhich said input circuit means further includes block signal generatingmeans for generating a block signal in response to a selected condition,said address control circuit being disabled in response to the receiptof a block signal.
 19. A routing stage as defined in claim 18 in whicheach said address control circuit includes:A. block signal receivingmeans connected to said input circuit means for receiving said blocksignal; B. address decoding means connected to said input circuit meansfor generating an address decode signal in response to a selecteddecoding of the address from said input circuit means; C. addressenabling signal generating means connected to said block signalreceiving means and said address decoding means for generating saidaddress enabling signal in response to the receipt of said addressdecode signal and the absence of said block signal.
 20. A routing stageas defined in claim 18 in which said address includes a protocol bit,said input circuit including protocol means for detecting said protocolbit and for generating a miss signal if said protocol bit it notdetected, said block signal generating means being connected to saidprotocol means for generating said block signal in response to thegeneration of said miss signal.
 21. A routing stage as defined in claim18 in which said each input circuit means is connected to a messagetransfer wire for receiving a message thereover, said input circuitincluding broken wire detection means for detecting the condition of themessage transfer wire and generating a broken wire signal in responsethereto, said block signal generating means being connected to saidbroken wire detection means for generating said block signal in responseto the generation of said broken wire signal.
 22. A routing stage asdefined in claim 21 in which each said output terminal means is alsoconnected to a message transfer wire, said message transfer wirestransferring messages between said output terminal means and said inputmeans, said output terminal means also including broken wire detectionmeans,A. each said broken wire detection means including:i. broken wiresignal generating means connected to said message transfer wire fortransmitting a signal over said message transfer wire; ii. broken wirelatch means connected to said message transfer wire for latching asignal received over said message transfer wire and generate said brokenwire signal in response to the condition of the latched signal; and B.said routing network further including broken wire control meansconnected to said broken wire signal generating means and said brokenwire latch means in each broken wire detection means for separatelyenabling said broken wire signal generating means in said input meansand said output terminal means to transmit a signal on the messagetransfer line, and the broken wire latch means in the other of inputmeans and said output terminal means to latch the condition of thesignal on the message transfer line.
 23. A routing stage as defined inclaim 13 in which said address comprises a predetermined number ofserial address bit signals, and wherein said address control circuits ineach circuit means are connected in series ordered relationship for thecircuit means' input circuit means, each address control circuitincluding:A. address bit signal receiving means for receiving an addressbit signal from said input circuit or from another address controlcircuit in said series order; B. address bit complement means connectedto said address bit signal receiving means for generating a complementaddress bit signal in response to the address bit signal received bysaid address bit signal receiving means; C. multiplexer means connectedto said address bit signal receiving means and said address bitcomplement means for selectively coupling either the address bit signalor the complement address bit signal to the next address control circuitin the series order; and D. control means connected to said address bitsignal receiving means and said multiplexer means for controlling saidmultiplexer means, said control means including:i. address bit signalcoupling means connected to said address bit signal receiving means forcoupling said address bit signal received by said address bit signalreceiving means; ii. latch means connected to said address bit signalcoupling means and said multiplexer means for latching the address bitsignal received from said address bit signal coupling means and forcontrolling said address bit signal coupling means and said multiplexermeans in response thereto, said latch means controlling said address bitsignal coupling means to couple address bit signals thereto while saidaddress bit signals have a selected condition and contemporaneouslyenabling said multiplexer means to couple the complement address bitsignal to the next address control circuit, and to disable said addressbit signal coupling means on receipt of an address bit signal havinganother condition and contemporaneously enabling said multiplexer meansto couples said address bit signal to the next address control circuit.24. A routing stage as defined in claim 13 for transferring messageseach including an acknowledge bit in response to an acknowledge transferenabling signal, each output terminal means includes means connected toits data line and an output terminal for coupling data signals from saiddata line to said output terminal, and output gate means connected tosaid output terminal and said data line for coupling said acknowledgebit from said output terminal to said data line in response to saidacknowledge transfer enabling signal, each of said input circuit meansincluding input gate means connected to said data output terminal andfor coupling said acknowledge bit from said data output terminal to saidinput terminal in response to the receipt of said acknowledge transferenabling signal.
 25. A routing network for transferring messages among aplurality of processing elements in an array processing system, saidrouting network switching messages in a plurality of initial switchingstages and terminating in a crossbar switch stage, each processingelement transmitting an address and data with each message, each of saidinitial switching stages including:A. a plurality of output terminalgroups each comprising a plurality of output terminal means, each outputterminal group being identified by a decoding of a selected portion ofan address, and each output terminal means includes means for generatinga coupling enabling signal in response to a selected condition; B. aplurality of data lines each connected to an output terminal means; C.circuit means for receiving a message and coupling it onto a selectedone of said data lines, said circuit means including:i. input circuitmeans for receiving an input message and having an address outputterminal for transmitting said address and a data output terminal fortransmitting said data; ii. a plurality of address control circuits fordecoding the address from address output terminal of said input circuitmeans, each address control circuit generating an address enablingsignal in response to a predetermined encoding of the address; and iii.a plurality of coupling means groups each connected to an addresscontrol circuit for receiving an enabling signal therefrom, eachcoupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data from said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means, wherein said coupling means within saidcoupling means group are interconnected in a predetermined order, with ahigh order coupling means being connected to receive said addressenabling signal from said address control circuit, and each couplingmeans being connected to transmit the address enabling signal to thenext lower order coupling means if it does not couple said data fromsaid input circuit means onto a data line, each coupling meansincluding:a. coupling enabling signal receiving means for receiving acoupling enabling signal; b. data coupling means responsive to thereceipt of a coupling enabling signal and an address enabling signal forcoupling data from said input circuit means onto a data line; and c.coupling enabling signal transmitting means connected to said couplingenabling signal receiving means and said data coupling means fortransmitting the coupling enabling signal to another coupling means inresponse to the receipt of a coupling enabling signal if said datacoupling mean does not couple data from said input circuit means on to adata line.
 26. A routing network as defined in claim 25 in which saiddata coupling means comprises:A. pass transistor means connected to saidinput circuit means and said data line for transferring data betweensaid input circuit means' data output terminal and said data line inresponse to a pass transistor control signal; B. control signalgenerating means connected to receive said coupling enabling signal andsaid address enabling signal for generating said pass transistor controlsignal in response to the coincidence of said address enabling signaland said coupling enabling signal.
 27. A routing network as defined inclaim 26 in which said coupling means further includes coincidence meansconnected to receive said coupling enabling signal and said addressenabling signal for transmitting an address enabling signal to said nextlower order coupling means in response to the coincidence of theassertion of said address enabling signal and the negation of saidcoupling enabling signal.
 28. A routing network for transferringmessages among a plurality of processing elements in an array processingsystem, said routing network switching messages in a plurality ofinitial switching stages and terminating in a crossbar switch stage,each processing element transmitting an address and data with eachmessage, said address including a protocol bit, each of said initialswitching stages including:A. a plurality of output terminal groups eachcomprising a plurality of output terminal means, each output terminalgroup being identified by a decoding of a selected portion of anaddress; B. a plurality of data lines each connected to an outputterminal means; C. circuit means for receiving a message and coupling itonto a selected one of said data lines, said circuit means including:i.input circuit means for receiving an input message and having an addressoutput terminal for transmitting said address and a data output terminalfor transmitting said data, said input circuit means further includes:a.block signal generating means for generating a block signal in responseto a selected condition; b. protocol means for detecting said protocolbit and for generating a miss signal if said protocol bit is notdetected, said block signal generating means being connected to saidprotocol means for generating said block signal in response to thegeneration of said miss signal; ii. a plurality of address controlcircuits for decoding the address from address output terminal of saidinput circuit means, each address control circuit generating an addressenabling signal in response to a predetermined encoding of the address,said address control circuits being disabled in response to the receiptof a block signal; and iii. a plurality of coupling means groups eachconnected to an address control circuit for receiving an enabling signaltherefrom, each coupling means group including a plurality of couplingmeans each connected to a data line, one of said coupling means within acoupling means group coupling said data from said input circuit meansonto a data line in response to the receipt of said address enablingsignal from said address circuit means.
 29. A routing network fortransferring messages among a plurality of processing elements in anarray processing system, said routing network switching messages in aplurality of initial switching stages and terminating in a crossbarswitch stage, each processing element transmitting an address and datawith each message, each of said initial switching stages including:A. aplurality of output terminal groups each comprising a plurality ofoutput terminal means, each output terminal group being identified by adecoding of a selected portion of an address; B. a plurality of datalines each connected to an output terminal means; C. circuit means forreceiving a message and coupling it onto a selected one of said datalines, said circuit means including:i. input circuit means for receivingan input message and having an address output terminal for transmittingsaid address and a data output terminal for transmitting said data, saideach input circuit means is connected to a message transfer wire forreceiving a message thereover, said input circuit means furtherincludes:a. block signal generating means for generating a block signalin response to a selected condition; b. broken wire detection means fordetecting the condition of the message transfer wire and generating abroken wire signal is response thereto, said block signal generatingmeans being connected to said broken wire detection means for generatingsaid block signal in response to the generation of said broken wiresignal; ii. a plurality of address control circuits for decoding theaddress from address output terminal of said input circuit means, eachaddress control circuit generating an address enabling signal inresponse to a predetermined encoding of the address, said addresscontrol circuit being disabled in response to the receipt of a blocksignal; and iii. a plurality of coupling means groups each connected toan address control circuit for receiving an enabling signal therefrom,each coupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data from said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means.
 30. A routing network as defined in claim 29in which each said output terminal means is also connected to a messagetransfer wire, said message transfer wires transferring messages betweensaid output terminal means of one of said initial switching stages andsaid input means of succeeding one of said initial switching stages,said output terminal means also including broken wire detection means,A.each said broken wire detection means including:i. broken wire signalgenerating means connected to said message transfer wire fortransmitting a signal over said message transfer wire; ii. broken wirelatch means connected to said message transfer wire for latching asignal received over said message transfer wire and generate said brokenwire signal in response to the condition of the latched signal; and B.said routing network further including broken wire control meansconnected to said broken wire signal generating means and said brokenwire latch means in each broken wire detection means for separatelyenabling said broken wire generating means in said input means and saidoutput terminal means to transmit a signal on the message transfer line,and the broken wire latch means in the other of input means and saidoutput terminal means to latch the condition of the signal on themessage transfer line.
 31. A routing network for transferring messagesamong a plurality of processing elements in an array processing system,said routing network switching messages in a plurality of initialswitching stages and terminating in a crossbar switch stage, eachprocessing element transmitting an address and data with each message,each of said initial switching stages including:A. a plurality of outputterminal groups each comprising a plurality of output terminal means,each output terminal group being identified by a decoding of a selectedportion of an address; B. a plurality of data lines each connected to anoutput terminal means; C. circuit means for receiving a message andcoupling it onto a selected one of said data lines, said circuit meansincluding:i. input circuit means for receiving an input messages andhaving an address output terminal for transmitting said address and adata output terminal for transmitting said data; ii. a plurality ofaddress control circuits for decoding the address from address outputterminal of said input circuit means, each address control circuitgenerating an address enabling signal in response to a predeterminedencoding of the address, wherein said address comprises a predeterminednumber of serial address bit signals, and wherein said address controlcircuits in each circuit means are connected in series orderedrelationship from the circuit means' input circuit means, each addresscontrol circuit including:a. address bit signal receiving means forreceiving an address bit signal from said input circuit or from anotheraddress control circuit in said series order; b. address bit signalcomplement means connected to said address bit signal receiving meansfor generating a complement address bit signal in response to theaddress bit signal received by said address bit signal receiving means;c. multiplexer means connected to said address bit signal receivingmeans and said address bit complement means for selectively couplingeither the address bit signal or the complement address bit signal tothe next address control circuit in the series order; and d. controlmeans connected to said address bit signal receiving means and saidmultiplexer means for controlling said multiplexer means, said controlmeans including:(i.) address bit signal coupling means connected to saidaddress bit signal receiving means for coupling said address bit signalreceived by said address bit signal receiving means; (ii.) latch meansconnected to said address bit signal coupling means and said multiplexermeans for latching the address bit signal received from said address bitsignal coupling means and for controlling said address bit signalcoupling means and said multiplexer means in response thereto, saidlatch means controlling said address bit signal coupling means to coupleaddress bit signals thereto while said address bit signals have aselected condition and contemporaneously enabling said multiplexer meansto couple the complement address bit signal to the next address controlcircuit, and to disable said address bit signal coupling means onreceipt of an address bit signal having another condition andcontemporaneously enabling said multiplexer means to couple said addressbit signal to the next address control circuit; (iii.) a plurality ofcoupling means groups each connected to an address control circuit forreceiving an enabling signal therefrom, each coupling means groupincluding a plurality of coupling means each connected to a data line,one of said coupling means within a coupling means group coupling saiddata from said input circuit means onto a data line in response to thereceipt of said address enabling signal from said address circuit means.32. A routing network for transferring messages among a plurality ofprocessing elements in an array processing system, said routing networkswitching messages in a plurality of initial switching stages andterminating in a crossbar switch stage, each processing elementtransmitting an address and data with each message, each of said initialswitching stages including:A. a plurality of output terminal groups eachcomprising aplurality of output terminal means, each output terminalgroup being identified by a decoding of a selected portion of anaddress; B. a plurality of data lines each connected to an outputterminal means; C. circuit means for receiving a message and coupling itonto a selected one of said data lines, said circuit means including:i.input circuit means for receiving an input message and having an addressoutput terminal for transmitting said address and a data output terminalfor transmitting said data; ii. a plurality of address control circuitsfor decoding the address from address output terminal of said inputcircuit means, each address control circuit generating an addressenabling signal in response to a predetermined encoding of the address;and iii. a plurality of coupling means groups each connected to anaddress control circuit for receiving an enabling signal therefrom, eachcoupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data from said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means; wherein each said message includes anacknowledge bit in response to an acknowledge transfer enabling signal,each output terminal means includes means connected to its data line andan output terminal for coupling data signals from said data line to saidoutput terminal, and output gate means connected to said output terminaland said data line for coupling said acknowledge bit from said outputterminal to said data line in response to said acknowledge transferenabling signal, each of said input circuit means including input gatemeans connected to said data output terminal and for coupling saidacknowledge bit from said data output terminal to said input terminal inresponse to the receipt of said acknowledge transfer enabling signal.33. A routing stage for transferring messages among a plurality ofprocessing elements in a routing network, the routing stage including:A.a plurality of output terminal groups each comprising a plurality ofoutput terminal means, each output terminal group being identified by adecoding of a selected portion of an address, and each output terminalmeans includes means for generating a coupling enabling signal inresponse to a selected condition; B. a plurality of data lines eachconnected to an output terminal means; C. circuit means for receiving amessage and coupling it onto a selected one of said data lines, saidcircuit means including:i. input circuit means for receiving an inputmessage and having an address output terminal for transmitting saidaddress and a data output terminal for transmitting said data; ii. Aplurality of address control circuits for decoding the address fromaddress output terminal of said input circuit means, each addresscontrol circuit generating an address enabling signal in response to apredetermined encoding of the address; and iii. a plurality of couplingmeans groups each connected to an address control circuit for receivingan enabling signal therefrom, each coupling means group including aplurality of coupling means each connected to a data line, one of saidcoupling means within a coupling means group coupling said data fromsaid input circuit means onto a data line in response to the receipt ofsaid address enabling signal from said address circuit means, whereinsaid coupling means within said coupling means group are interconnectedto a predetermined order, with a high order coupling means beingconnected to receive said address enabling signal from said addresscontrol circuit, and each coupling means being connected to transmit theaddress enabling signal to the next lower order coupling means if itdoes not couple said data from said input circuit means onto a dataline, each coupling means including:a. coupling enabling signalreceiving means for receiving a coupling enabling signal; b. datacoupling means responsive to the receipt of a coupling enabling signaland an address enabling signal for coupling data from said input circuitmeans onto a data line, said data coupling means comprises:(i.) passtransistor means connected to said input circuit means and said dataline for transferring data between said input circuit menas' data outputterminal and said data line in response to a pass transistor controlsignal; (ii.) control signal generating means connected to receive saidcoupling enabling signal and said address enabling signal for generatingsaid pass transistor control signal in response to the coincidence ofsaid address enabling signal and said coupling enabling signal; c.coupling enabling signal transmitting means connected to said couplingenabling signal receiving means and said data coupling means fortransmitting the coupling enabling signal to another coupling means inresponse to the receipt of a coupling enabling signal if said datacoupling means does not couple data from said input circuit means onto adata line.
 34. A routing network as defined in claim 33 in which saidcoupling means further includes coincidence means connected to receivesaid coupling enabling signal and said address enabling signal fortransmitting an address enabling signal to said next lower ordercoupling means in response to the coincidence of an assertion of saidaddress enabling signal and the negation of said coupling enablingsignal.
 35. A routing stage for transferring messages among a pluralityof processing elements in a routing network, the routing stageincluding:A. a plurality of output terminal groups each comprising aplurality of output terminal means, each output terminal group beingidentified by a decoding of a selected portion of an address, saidaddress including a protocol bit; B. a plurality of data lines eachconnected to an output terminal means; C. circuit means for receiving amessage and coupling it onto a selected one of said data lines, saidcircuit means including:i. input circuit means for receiving an inputmessage and having an address output terminal for transmitting saidaddress and a data output terminal for transmitting said data said inputcircuit means further includes:a. block signal generating means forgenerating a block signal in response to a selected condition; b. aprotocol means for detecting said protocol bit and for generating a misssignal if said protocol bit it not detected, said block signalgenerating means being connected to said protocol means for generatingsaid block signal in response to the generation of said miss signal; ii.a plurality of address control circuits for decoding the address fromaddress output terminal of said input circuit means, each addresscontrol circuit generating an address enabling signal in response to apredetermined encoding of the address said address control circuit beingdisabled in response to the receipt of a block signal; and iii. aplurality of coupling means groups each connected to an address controlcircuit for receiving an enabling signal therefrom, each coupling meansgroup including a plurality of coupling means each connected to a dataline, one of said coupling means within a coupling means group couplingsaid data from said input circuit means onto a data line in response tothe receipt of said address enabling signal from said address circuitmeans.
 36. A routing stage for transferring messages among a pluralityof processing elements in a routing network, the routing stageincluding:A. a plurality of output terminal groups each comprising aplurality of output terminal means, each output terminal group beingidentified by a decoding of a selected portion of an address; B. aplurality of a data lines each connected to an output terminal means; C.circuit means for receiving a message and coupling it onto a selectedone of said data lines, said circuit means including:i. input circuitmeans for receiving an input message and having an address outputterminal for transmitting said address and a data output terminal fortransmitting said data said each input circuit means is connected to amessage transfer wire for receiving a message thereover, said inputcircuit means further includes:a. block signal generating means forgenerating a block signal in response to a selected condition; b. brokenwire detection means for detecting the condition of the message transferwire and generating a broken wire signal in response thereto, said blocksignal generating means being connected to said broken wire detectionmeans for generating said block signal in response to the generation ofsaid broken wire signal; ii. a plurality of address control circuits fordecoding the address from address output terminal of said input circuitmeans, each address control circuit generating an address enablingsignal in response to a predetermined encoding of the address, saidaddress control circuit being disabled in response to the receipt of ablock signal; and iii. a plurality of coupling means groups eachconnected to an address control circuit for receiving an enabling signaltherefrom, each coupling means group including a plurality of couplingmeans each connected to a data line, one of said coupling means within acoupling means group coupling said data from said input circuit meansonto a data line in response to the receipt of said address enablingsignal from said address circuit means.
 37. A routing stage as definedin claim 36 in which each said output terminal means is also connectedto a message transfer wire, said message transfer wires transferringmessages between said output terminal means and said input means, saidoutput terminal means also including broken wire detection means,A. eachsaid broken wire detection means including:i. broken wire signalgenerating means connected to said message transfer wire fortransmitting a signal over said message transfer wire; ii. broken wirelatch means connected to said message transfer wire for latching asignal received over said message transfer wire and generate said brokenwire signal in response to the condition of the latched signal; and B.said routing network further including broken wire control meansconnected to said broken wire signal generating means and said brokenwire latch means in reach broken wire detection means for separatelyenabling said broken wire signal generating means in said input meansand said output terminal means to transmit a signal on the messagetransfer line, and the broken wire latch means in the other of inputmeans and said output terminal means to latch the condition of thesignal on the message transfer line.
 38. A routing stage fortransferring messages among a plurality of processing elements in arouting network, the routing stage including:A. a plurality of outputterminal groups each comprising a plurality of output terminal means,each output terminal group being identified by a decoding of a selectedportion of an address; B. a plurality of data lines each connected to anoutput terminal means; C. circuit means for receiving a message andcoupling it onto a selected one of said data lines, said circuit meansincluding:i. input circuit means for receiving an input message andhaving an address output terminal for transmitting said address and adata output terminal for transmitting said data; ii. a plurality ofaddress control circuits for decoding the address from address outputterminal of said input circuit means, each address control circuitgenerating an address enabling signal in response to a predeterminedencoding of the address, wherein said address comprises a predeterminednumber of serial address bit signals, and wherein said address controlcircuits in each circuit means are connected in series orderedrelationship form the circuit means' input circuit means, each addresscontrol circuit including:a. address bit signal receiving means forreceiving an address bit signal from said input circuit or from anotheraddress control circuit in said series order; b. address bit signalcomplement means connected to said address bit signal receiving meansfor generating a complement address bit signal in response to theaddress bit signal received by said address bit signal receiving means;c. multiplexer means connected to said address bit signal receivingmeans and said address bit complement means for selectively couplingeither the address bit signal or the complement address bit signal tothe next address control circuit in the series order; and d. controlmeans connected to said address bit signal receiving means and saidmultiplexer means for controlling said multiplexer means, said controlmeans including:(i.) address bit signal coupling means connected to saidaddress bit signal receiving means for coupling said address bit signalreceived by said address bit signal receiving means; (ii.) latch meansconnected to said address bit signal coupling means and said multiplexermeans for latching the address bit signal received from said address bitsignal coupling means and for controlling said address bit signalcoupling means and said multiplexer means in response thereto, saidlatch means controlling said address bit signal coupling means to coupleaddress bit signals thereto while said address bit signals have aselected condition and contemporaneously enabling said multiplexer meansto couple the complement address bit signal to the next address controlcircuit, and to disable said address bit signal coupling means onreceipt of an address bit signal having another condition andcontemporaneously enabling said multiplexer means to couple said addressbit signal to the enxt address control circuit; (iii.) a plurality ofcoupling means groups each connected to an address control circuit forreceiving an enabling signal therefrom, each coupling means groupincluding a plurality of coupling means each connected to a data line,one of said coupling means within a coupling means group coupling saiddata from said input circuit means onto a data line in response to thereceipt of said address enabling signal from said address circuit means.39. A routing stage for transferring messages among a plurality ofprocessing elements in a routing network, the routing stage including:A.a plurality of output terminal groups each comprising a plurality ofoutput terminal means, each output terminal group being identified by adecoding of a selected portion of an address; B. a plurality of datalines each connected to an output terminal means; C. circuit means forreceiving a message and coupling it onto a selected one of said datalines, said circuit means including:i. input circuit means for receivingan input message and having an address output terminal for transmittingsaid address and a data output terminal for transmitting said data; ii.a plurality of address control circuits for decoding the address fromaddress output terminal said input circuit means, each address controlcircuit generating an address enabling signal in response to apredetermined encoding of the address; and iii. a plurality of couplingmeans groups each connected to an address control circuit for receivingan enabling signal therefrom, each coupling means group including aplurality of coupling means each connected to a data line, one of saidcoupling means within a coupling means group coupling said data fromsaid input circuit means onto a data line in response to the receipt ofsaid address enabling signal from said address circuit means; whereineach said message includes an acknowledge bit in response to anacknowledge transfer enabling signal, each output terminal meansincludes means connected to its data line and an output terminal forcoupling data signals from said data line to said output terminal, andoutput gate means connected to said output terminal and said data linefor coupling said acknowledge bit from said output terminal to said dataline in response to said acknowledge transfer enabling signal, each ofsaid input circuit means including input gate means connected to saiddata output terminal and for coupling said acknowledge bit from saiddata output terminal to said input terminal in response to the receiptof said acknowledge transfer enabling signal.
 40. A routing stage fortransferring messages among a plurality of processing elements in arouting network, the routing stage including:A. a plurality of outputterminal groups each comprising a plurality of output terminal means,each output terminal group being identified by a decoding of a selectedportion of an address, and each output terminal means includes means forgenerating a coupling enabling signal in response to a selectedcondition; B. a plurality of data lines each connected to an outputterminal means; C. circuit means for receiving a message and coupling itonto a selected one of said data lines, said circuit means including:i.input circuit means for receiving an input message and having an addressoutput terminal for transmitting said address and a data output terminalfor transmitting said data; ii. a plurality of address control circuitsfor decoding the address from address output terminal of said inputcircuit means, each address control circuit generating an addressenabling signal in response to a predetermined encoding of the address;and iii. a plurality of coupling means groups each connected to anaddress control circuit for receiving an enabling signal therefrom, eachcoupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data from said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means, wherein said coupling means within saidcoupling means group are interconnected in a predetermined order, with ahigh order coupling means being connected to received said addressenabling signal from said address control circuit, and each couplingmeans being connected to transmit the address enabling signal to thenext lower order coupling means if it does not couple said data fromsaid input circuit means onto a data line, each coupling meansincluding:a. coupling enabling signal receiving means for receiving acoupling enabling signal; b. data coupling means responsive to thereceipt of a coupling enabling signal and an address enabling signalfrom coupling data from said input circuit means onto a data line; andc. coupling enabling signal transmitting means connected to saidcoupling enabling signal receiving means and said data coupling meansfor transmitting the coupling enabling signal to another coupling meansin response to the receipt of a coupling enabling signal if said datacoupling means does not couple data from said input circuit means onto adata line.
 41. A routing network for transferring messages among aplurality of processing elements in an array processing system, saidrouting network switching messages in a plurality of initial switchingstages and terminating in a crossbar switch stage, each processingelement transmitting an address and data with each message, each of saidinitial switching stages including:A. a plurality of output terminalgroups each comprising a plurality of output terminal means, each outputterminal group being identified by a decoding of a selected portion ofan address, and each output terminal means includes means for generatinga coupling enabling signal in response to a selected condition; B. aplurality of data lines each connected to an output terminal means; C.circuit means for receiving a message and coupling it onto a selectedone of said data lines, said circuit means including:i. input circuitmeans for receiving an input message and having an address outputterminal for transmitting said address and a data output terminal fortransmitting said data; ii. a plurality of address control circuits fordecoding the address from address output terminal of said input circuitmeans, each address control circuit generating an address enablingsignal in response to a predetermined encoding of the address; and iii.a plurality of coupling means groups each connected to an addresscontrol circuit for receiving an enabling signal therefrom, eachcoupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data rom said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means, wherein said coupling means within saidcoupling means group are interconnected in a predetermined order, with ahigh order coupling means being connected to receive said addressenabling signal from said address control circuit, and each couplingmeans being connected to transmit the address enabling signal to thenext lower order coupling means if it does not couple said data fromsaid input circuit means onto a data line.
 42. A routing stage fortransferring messages among a plurality of processing elements in arouting network, the routing stage including:A. a plurality of outputterminal groups each comprising a plurality of output terminal means,each output terminal group being identified by a decoding of a selectedportion of an address, and each output terminal means includes means forgenerating a coupling enabling signal in response to a selectedcondition; B. a plurality of data lines each connected to an outputterminal means; C. circuit means for receiving a message and coupling itonto a selected one of said data lines, said circuit means including:i.input circuit means for receiving an input message and having an addressoutput terminal for transmitting said address and a data output terminalfor transmitting said data; ii. a plurality of address control circuitsfor decoding the address from address output terminal of said inputcircuit means, each address control circuit generating an addressenabling signal in response to a predetermined encoding of the address;and iii. a plurality of coupling means groups each connected to anaddress control circuit for receiving an enabling signal therefrom, eachcoupling means group including a plurality of coupling means eachconnected to a data line, one of said coupling means within a couplingmeans group coupling said data from said input circuit means onto a dataline in response to the receipt of said address enabling signal fromsaid address circuit means, wherein said coupling means within saidcoupling means group are interconnected in a predetermined order, with ahigh order coupling means being connected to receive said addressenabling signal from said address control circuit, and each couplingmeans being connected to transmit the address enabling signal to thenext lower order coupling means if it does not couple said data fromsaid input circuit means onto a data line.